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ABSTRACT J 

' Based on a workshop on vision research issues in 

Vf light-»training simulators held in June 1980, this report focuses on 
Approaches for the conduct of research on what visual information is 
needed for simulation and how it can best be presented. An 
introduction gives an overview of the workshop and describes the 
contents^ of the report. Section 1 discusses the general need for 
research in the' area and suggests ways to facilitate the research 
needed and to stimulate interaction between scientists and simulation 
researchers. Section 2 describes some technical problems in more 
detail and gives a few examples of applying research strategies to 
these problems. Fitte appendix papers provide examples of research 
strategies. Primary emphasis is given to low-level flight, in which 
extraction of visual information from terrain features. is crucial but 
little understood. Values and pitfalls are described of these 
strategies for exploring which visual features should be used in 
low-level flight: systemat ic condensation of opinions, particularly 
of pilots (appendix A); geometric analysis of terrain information 
potentially usable (B); and psychophysical analysis of visual 
processing modalities (C and D) . Appendix E examines equipment * 
requirements for display of visual information. Appendix F lists 
workshop topics and participants. (TLB) 
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, SUMMARY 



Thia effort bid two objectivea. The first objective wu to promote interaction between vision-research 
•dentist, and pertonnel working on flight simulator reaearch and development (R&D). The second objective wu ■ 
set of reeoinmendations regarding R&D needa and strategies that might be useful in developing more effective 
displays of viaual acenea in flight simulation, particularly for low-level flight. 

Background 

In general, the development and use of visual displays in flight simulation have been dominated by 
engineering concerns. Equipment for simulating visual scenes haa become highly sophisticated, but vision science 
haa played a limited role in determining what visual information is presented in simulators and how. As a result, 
fundamental questions remain unanswered as to how to make the visual displaya used in flight simulators most 
effective or appropriate for flight training. 

Approach 

A workshop was organized by the Committee on Vision of the National Research Council and the Operations 
Training Division of the Air Force Human Resources Laboratory (AFHRL/OT), Williams AFB, Ariaona. Tina 
workshop wu held in June 1980 at Williams AFB and was attended by vision scientists from the academic 
community, u weU u personnel from industry and the military services who are engaged in RaD on visual 
displays for flight simulation. Following the workshop, members of the Committee on Vision identified key HAD 
issue* and strategies. 

Specifics 

The two-day workshop began with presentations to familiarise participants with the visual simulation RaD at 
AFHRL/OT This wu followed by a tour of the wide-field-of-view computer image generation facility there. A 
second set of presentations informed the AFHRL/OT staff about complementary research activities in portions of 
the academic community. 

A steering group then developed examples of specific R&D approaches that might help clarify several of the 
long-range issues confronted in visual simulation. Primary emphasis wu given to low-level flight, in wlucli the 
extraction of visual information from terrain features is crucial but not well understood. Several strategies were 
•attested for exploring what visual features may be used in low-level flight. Specifically discussed were: W 
systematic condensation of opinions, particularly those of pilots, (b) geometric analysis of terrain information, and 
(c) peyebophysical analyses of visual processing modalities. 



Conclnalona/Raconunendations 

The workahop format promoted productive technical interactions among llie-participanta. It was concluded 
lll»t multiple RAD lines were iieeded to anawer the fundamental questions about visual display requirements for 
flight simulation. A mixture of R&D approaches wu recommended, ranging from short-term attack* on issues of 
immediate operational concern, to long-range investigations of fundamental iasues. Need for greater attention to 
training issues wu noted, since in many case, the best R&D approach to viaual problem, cannot be determined 
until training needs are specified. 



PREFACE , 



The work report herein was accomplished in support of .he Air Force Hum.. Resources Laboratory 
Air J!T .nd Training Research Thrus,. In order .o effectively utilize fhgh, ..mutator, to suppor 

.dv.n ed .ir comb., tr.ining requirements, signific.n, improvements in the display of ou.-of-cockpi. fhgh, 
„v^„,en« will he required, fhe purpose of ,his work »ni, w.sio focus 
on the design of rese.rch strategies .ppropri.,e for resolution of major issues .n visual s,mul..,on 
requirements. 

Thin repor, is based on a workshop held in June 1980 ,o discuss problems encounlered in visual 
sulfation of flighl and ,o explore ways of stimulating better research in this area. Low-leve) flight was 
chosen as a problem example for discussion of how various research strategies might be applied. This limited 
effort is not intended to be a comprehensive analysis of the problems encountered in visual simulation of 
flighl. and there has been no attempt to lay out priorities for research. Such a comprehensive assessment 
would be an enormous undertaking requiring a much more broadly constituted group. Rather, the approach , . 
has been to attempt to give a sense of the complicated mixture of issues encountered in detegnining what 
visual display information is required for effective simulation of a given flight environment. Examples are 
presented to show how basic research perspectives might be used to attack these issues. 

Individual research perspectives of some members of the steering group are included as appendixes to 
(his report The body of the report discusses general issues, suggests ways to facilitate interaction of basic 
scientist and simulation engineers, and attempts to show the complementarity of several approaches to 
research. The entire group discussed the body of the report, written by Whitman Richards with the assistance 
of Kry DinmukcH. 

Robert Kennedy of the National Research Council and Stanley Collyer of the Naval Training Equipment 
Center substantially assisted the steering group with numerous discussions, suggestions, and ident.ficat.on of 
relevant literature. With some effort, they provided the steering group a quick education in the practical 
problems of simulation. George Buckland, the Air Force project officer for this study, played a major role 
oVgani/.ing the workshop on which this repor. is based. Lloyd Kaufman and Conrad Kraft made helpfu 
comments on an early draft of this report and provided thoughtful suggestions about the problems of visual 
Hirnulation n'Hearcli. ^r 0 ^ 

The project that is the subject of this report was approved by the Governing Board of the Nafmnal 
Research Council, whose members arc drawn from the councils of the National Academy of Sciences, the , 
National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible 
for the report were chosen for their special competences and with regard for appropriate balance. 

This report has been reviewed by a group other than the authors according to procedures approved by a 
Report Review Committee consisting of members of the National Academy of Sciences, the National Academy 
of Engineering, and the Institute of Medicine. 

The National Research Council was established by the Nation.r Academy of Sciences in 1916 to associate 
the broad community of science and technology with the Academy's purposes of furthering knowledge and 
of advising the federal government. The Council operates in accordance with genera polices determined by 
the Academy under the authority of its congressional charter of 1863, which establishes the Academy as a 
private, nonprofit, self-governing membership corporation. The Council has become the principal-operating 
agency of both the National Academy of Sciences and the National Academy of Engineering ,n the conduct of 
their services to the government, the public, and the scientific and engineering communities. It is 
administered jointly by both Academies and the Institute of Medicine. The National Acaden,y."&f Engineering 
and the Institute of Medicine were established in 1964%nd 1970, respectively, under the charter of the 
National Academy of Srienra. 
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VISION RESEARCH FOR FLIGHT SIMULATION 



INTRODUCTION 



The U.S. Air Force Human Resources Laboratory (USAFHRL) asked the 
Committee on Vision to organize a workshop on vision research issues in 
flight-training simulators/particularly those using computer image 
\geneS£tion (CIG) techniques. The workshop, held at Williams Air Force 
Jase, Ariz., in June 1980, had two purposes. First, it was an experi- 
ent in facilitating interaction between vision scientists and person- 
nel working on flight Simulator research. Second, it explored research 
ieeds and strategies that might be useful in developing more effective 
Lsplay of visual scenes, particularly for low-level flight.. 1 . 
i. . In general, the development and use of visual displays iW flight 
simulation have been dominated by engineering concerns. Equipment for 
simulating visual scenes has become highly sophisticated, but vision 
sckence has played a small role in determining- what visual information 
is presented in simulators and the way it is presented. Vision scien- 
tists have done limited work on determining the information required 
for effective visual Simulation. Well-controlled experiments, to com- 
pare training effectiveness of different approaches to visual simula- 
tion are expensive, difficult, and time consuming, partly because they 
cannot be readily conducted in laboratories. Little ' theoretical work 
is available to predict the kinds of visual information -that should be 
included in a particular simulation. Consequently, simulator designers 
and buyers have typically opted for as much "realism"2 and display capa- 
bility as possible, since the research literature does not provide a 
basis for selecting or evaluating simpler displays. ' - 

The two-day workshop (see appendix F for topics and participants) 
began with presentations to familiarize participants with visual simu- 
lation 1 research efforts at the Operations Training Division of USAFHRL 
at Williams AFB and was followed by a demonstration of *he wide-field 
CIG used for research there. A second set of presentations informed 
the Williams staff about complementary research activities in portions 



^ow-level flight is defined as flying along the contour of the earth, 
usually at an altitude below 200 feet (above ground level). 

2 Realism is used here to include both accuracy (fidelity to real-world 
characteristics) and completeness of detail. 
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of the academic coramx^ityi During the first day, it rapidly became 
obvious that an enormous number of research issues are involved in at- 
tempting to develop effective visual simulations. These issues in- 
clude CIG techniques, the capabilities of the human visual system, in- 
formation processing needs, attentional factors, measures of training 
effectiveness, and strategies for enhancing training (see table 1). 
But researchers h$ve made little progress in isolating components of 
major simulation issues so that they can be attacked separately. Also, 
training needs apparently have not been sufficiently well character- 
ized to allow visual requirements to be defined accordingly. This 
range of problems was clearly too large for a small group of vision 
scientists to address in depth in a single workshop. Thus, on the 
second day, the discussion concentrated on possible research strategies 
for determining whatr visual information is needed in a simulator for a 
'pilot to perform a /required flight task. The issues of training and 
*simuJbdtor evaluation were s«t aside, although participants continually 
noted that these issues are important and deserve careful consideration 
(see next section) . L 

Visual scene requirements differ considerably among flight tasks 
simulated, and research nepds will vary accordingly. A particular 
flight environment, low J level flight, was chosen as a problem-solving 
example for the second day of workshop discussion. This" choice wad 
based on several factors. Acquisition of basic ^flying skills, such as 
take-off and landing, has been shown to occur With simulation training,, 
even with fairly simple visual displays (Sejnple et al., 1980; Waaig, 
1981). Much less is known about the visual requirements for simulating 
advanced tasks such as low-level flight for either initial acquisition ^ 
or maintenance of skills. Low-level flight is an important Air Force 
mission, yet it has been* difficult 'to simulate adequately with CIG sys- 
tems. Even experienced pilots may misjudge altitude, range , and shape 
of the ground in simulators \ problems that suggest that some unknown 
aspects of the visual display are inadequate to allow performance^ com- 
parable to that in the real world. Low-level flight may be a worst- 
^^bse example In which visual demands are great and simulation require- 
ments are not understood/ Research approaches that help determine 
visual display requirements for low-level flight may also prove useful 
in analyzing visualrretjbirements fbr toher, simpler flight tasks. 

After the workshop, the steering \roup met several times to discuss 
research issues and prepare this report. We do not intend this limited 
effort to be a comprehensive analysis of the problems encountered in 
visual simulation of flight, and we have not attempted to establish re- 
search priorities. "Such a comprehensive assessment would be an enor- 
mous undertaking requiring a much more broadly constituted group. We 
have attempted to give a sense of the complicated mixture of issues 
encountered in attempting to determine the visual* information required 
for effective simulation of a given flight environment. This report 
has twcfc main parts. First, we will discuss the general need for 



/ 



However, see discussion fn the following section of whether performance 
in simulator^ is an adequate measure of training effectiveness. 



TABLE 1 Somfc Issues in Simulator Training Research 



Training & Management 
Issues 

Tasks for Simulation 
Training 

Training Strategies 

Initial Training vs. 
Skill Maintenance 

Whole vs. Part Task 
Training 



Ktfotfledge-Based 

Issues 



Instructional 
issues 



Hardware 
Requirements 



Role o^Instructor 
Task Difficult^. Level 



Augmentation of 'Feedback* 

Scheduling of Feedback 

Adaptive vs. Fixed 
Amount of Training 




Visual System 

i Motion Systems 

A/C Dynamics 

Instructional 
eatures 

t ion- Visual 
Synchronization 



Behavioral Issues 



Experimental 
Methodologies 

Performance 
Assessment 

Application of Basic 
Research Findings 



Individual Differences 
Attention and Workload 
^kill Degradation 
Motivation 
Components of Skills 
Perception 
CognitJ*^ 

Response Processes 1 



Visual & Display 
Issues 



Image Quality' 
Image Detail 
FOV Size 
Realism 




SOURCE: R. Hennessy , unpublished. 
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research in this area in a way we hope will cohvey the magnitude <^f the' 
problems encountered. In this context, we will suggest ways to facili- 
tate the research needed and to stimulate interaction between scien- 
tists and simulation researchers. Second, we will describe some of the* 
technical problems in more detail, giving a Tew examples of applying 
basic research strategies to these problems. We have identified some 
areas of research that may be relevant to the simulation problems; how- 
ever, these examples are not intended as a comprehensive list,. 

Individual authors developed the examples of research strategies 
as appendix papers for this report. We asked the authors to describe, 
preferably by example, the value and pitfalls of the particular re- 
search approaches they would pursue. These diverse approaches are 
complementary; # we will attempt to show in the body of the report how 
they might be combined to address some aspects of the problem of simu- 
lating low-level flight. Some authors developed their approaches*, from 
a particular theoretical perspective; we will btiefly discuss assump- 
tions underlying these perspectives, but we will not debate whether * 
they will eventually be proven correct. At this point in simulation 
research ; \ the value of such theoretical orientations is in helping 
formulate sharp questions that will generate useful data. 

Other examples of research approaches could have been chosen 
equally well for these illustrations . £here is no one best research 
approach to a field as large and uncertain as visual simulation, even 
if one concentrates on a particular task simulation. Rather, a diverse 
mixture of research approaches should be encouraged. Indeed, the mem~ 
bers of this small steering group represent a range of research per- 
spectives, and each would attack these problems differently . The choice 
of these research examples: is oriented toward the particular problems 
raised by low-level flight. For instance, several examples emphasize 
geometric aspects of tetrain. Some, or all of -these research approaches 
might also be used to study simulation o ftfother flight tasks; however, 
the emphasis would be different/ We stress that . effective research , 
strategies must include an analysis of. the particular features and re- 
quirements of the flight task to be simulated. / ' 

Simulation workers will note that most of these examples of re- 
search strategy differ from those typically found in simulation re- 
search literature. In contrast to simulation research focused on ex- 
isting equipment, these examples of basic research address long-range 
issues of visual simulation. We have emphasized the fundamental side 
because that is the aspect for which the Air Force has the fewest re- 
sources and in which we are best able to .assist. 



BACKGROUND: PROBLEMS IN VISUAL SIMULATION RESEARCH 



Simulators are widely used for commer£^al, military, and space 
flight training. Simulation training has been shown to reduce substan- 
tially the number of attual flying hours required for student pilots ^ 
to master basic skills^and for experienced pilots to learn about new 
aircraft. In addition to reducing training costs, simulators can be 
used to train pilots safely in hazardous maneuvers and to create situ- 
ations that do not occur or cannot be adequately controlled in the real 
world (Hennessy, 1981). 

kapid improvements in computer image generation (CIG) techniques 
in the past decade have resulted in fairly sophisticated visual dis- 
plays for simulation. Nevertheless, the contribution of visual dis- 
plays to overall training effectiveness of simulators, is poorly under- 
stood. A few studies have systematically examined effects of visual 
display features on training effectiveness (see, for .example, Semple 
et al., 1980; Waag, 1981). Concern about visual display effectiveness 
is being raised, particularly ^among military R&D personnel, because of. 
several developments, especially the following: 

1. The costs of advanced simulators are rising rapidly. Visual 
simulation has been pushed to provide all the realism and 

s capability afforded by rapid advances in technology, with 
attendant rises/Ln cost. 

2. Some researchers have questioned the value of realism as a 
guide f<?r visual display (see, for example, Coblitz, 1980; 
Hennessy et al., 1980). 

3. Simulation of advanced flight maneuvers and environments, 
sufch as low- level flight, has been found to be problematic. 
The adequacy of training in existing simulators for low-level 
flight and other visually demanding tasks is not clear from 
the few existing studi^ . , 

The development and use of visual displays in simulators have been 
dominated by engineering considerations. 5 Visual scientists and psy- 
chologists have played only a minor role in deciding what visual in- 
formation should be displayed and how. Design engineers have developed 
visual displays based on common sense and previous experience. The 
major criterion of adequacy of displays has been their degree of ac- 
ceptance by experienced pilots. The evaluations of both ^pilots and de- 
signers appear to be based largely on how realistic visual displays 
appear, rather than explicit consideration of training effectiveness. 

The contributions of visual scientists have been limited, largely 
because they have not been able to tell designers the kind of informa- 
tion that visually displayed scenes should contain. Our knowledge of 
higher-order perceptual processes is too limited to specify with 



*For more extensive discussions, see Brown (1976), Hennessy et al. 
(1980), and Semple et al. (1980). 

^For more extensive discussion of this issue, see Boff and Martin 
(1980), Fulgham (1978), an^Hennessy (1981) . 



certainty the forms in which visual information is extracted by pilots 
performing particular tasks in the real world. Uncertainty about both 
visual information processing and perceptual and cognitive factors in 
training bakes it difficult to devise coherent strategies for display 
of visual information, in simulators. So far, the contribution of vision 
science to simulation research has been primarily in the area of image 
quality (resolution, contrast, distortion, etc.). Psychophysical stud- 
ies (e.g., Kraft et al. j 1930) of human visual sensitivities have ^pro- 
vided information needed by engineers attempting to design displays to 
match human capabilities. 

Semple et al*. (1980) point put that because of this uncertainty 
about visual information requirements , it has .probably been reasonable 
for designers to emphasize development of visual displays with maximum 
da^ail and fidelity. No evidence, however, supports the common assump- 
tioh that increasing realism improves the training value of simulators 
(see also Waag, 1981). Furthermore, hfghly realistic display of ex- 
tended- terrain for missions such as low-level flight is not possible 
with existing CIG equipment, and there may be inherent limitations to 
display realism (see appendix B) . 

Training issues are beyond the scope of this report and the compe- 
tence of its authors; however, visual display issuea must be considered 
in the context of training requirements. Training sjWulators may be 
used for acquisition of basic flight skills by student pilots, for 
transition to other types of aircraft by experienced pilots, and for 
proficiency maintenance in a given aircraft. j Visual display require- 
ments may differ cdnsiderably among these three training roles and with 
different flight tasks. For example, acquisition of basic skills has 
been shown to be substantially enhanced by simulation training with 
simple and in some cases highly unrealistic visual displays (Hennessy^ 
et al., 1981). In contrast, proficiency maintenance of visually de- 
manding tasks might require more detailed visual presentation; however, 
in the absence of data we can only speculate on this point. 

Collyer has suggested (private communication) that three kinds of 
skill? are acquired in flight training: 6 

1. Perceptual: "When the world looks like this, what is the 
aircraft's situation?" 

2. Decision/procedural: "When the aircraft is in this situation, 
what needs to be done next?" 

3. Control: "How do I make it do that?" 

Traditionally, flight instruction has emphasized procedural and control 
skills; little attention has been given to perceptual skills, and in 
fact little is known about perc'eptual learning processes in flight 
training. It may be that the demonstrated effectiveness of simulators 
in training basic flighty skills results mainly from learning procedural 
and control skills, and in this case the degree of realism of the 
•visual display may not be very important. Perceptual learning might 
play a large- role in learning and/or maintaining proficiency of some 
flight tasks than in others. Resolution of this Issue would facilitate 



Also see Roscoe (1979) on this point. 



cost-effective design of simulators for specific tasks. Unfortunately, 
there has been little good research on this point*, although some recent ' 
studies appear promising (e.g., Coblitz , 1980; Hennessy et al., 1981). 

The ultimate measure of simulator training effectiveness is the 
enhancement of actual flight performance. Unfortunately, trans fer-of- 
training studies are difficult and costly, and they must compete with ~~ 
operational missions for pilots and equipment. Although transfer-en- 
training studies have demonstrated effectiveness of simulator training 
for a number of tasks, almost none of these studies has been designed - 
to reveal the contribution to training transfer of the different fea- 
tures of the equipment and simulation instruction (Waag, 1981). Thus 
we have virtually no empirical information on how transfer of simula- 
tion training is affected by visual display characteristics or by the 
Information content of the visual scene. Several investigators have 
suggested that instruction methods used in simulator training may have 
as much or more Impact on effectiveness as do equipment features (e.g., 
Caro, 1977). '\ , 

Trans fer-of -training studies that simultaneously evaluate ef fects - 
of several simulator equipment features, trainees f characteristics, and 
training procedure variables are urgently needed (Simon and Roscoe, 
1981). It would also help to have techniques for predicting Simulator 
training effectiveness, to be usedf-to determine which issues, procedures, 
and equipment warrant full-scale transfer studies. Pilots 1 performance 
in the simulator has been used as such a measure (e.g., Westra et'al., 
1981). °This measure, however, is problematic. Factors that affect 
performance in the simulator may not have the same effect on transfer 
of training. Platform motion, for instance, in some studies improved 
pilot performance in the simulator but did not affect transfer to actual 
^ffcfcght performance in these and other studies (e.g., Waag, 1981). It 
is sometimes assumed that factors that do not influence performance in 
ttje simulator will not contribute to transfer of training; however, this 
assumption needs careful analysis and experimental evaluation. 

A major problem with most transfer-of-training and performance I 
studies is that their designs do not allow generalizations to be draVm. 
In addition to not discriminating among effects of different Simula- V 
t ion factors, most studies have too narrow a focus (e.g., studies on a 
particular piece of equipment or a specific operational need) to shed 
light on the general questions about visual information requirements, 
these studies are not additive. In contrast, the work of Westra and 
coworkers (1981) and Simon and Roscoe (1981) , among others , suggests 
possibilities for developing more powerful* empirical methods of 
evaluation. 

In summary, evaluation studies have demonstrated that visual simu- 
lation can substantxWly enhance training of some (but not necessarily 
all) flight tasks. There is, however, little information on the rela- 
tive training effectiveness of different approaches to visual, informa- 
tion display. * Virtually nothing can be said with certainty about the 
best, way to display visual information for simulator training of par- 
ticular tasks. ^ 
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RESEARCH NEEDS 



As pointed out earlier, empirical studies of the relative training 
effectiveness of different approaches to visual Simula ti<afc t have been 
quite limited. On the basis of either existing (evaluation studies or 
theories of visual perception, it is impossible to specify with any 
certainty the visual information requirements for simulation training 
of specific flight tasks. Simulators in use today provide effective 
training of some, not all, flight tasks; however, it is important to 
consider cost effectiveness as well as training efficacy (Orlansky and 
String, 1977) • Lack of /knowledge of visual information display require- 
ments makes it difficult to maximize training effectiveness or minimize 
costs. . I 

Research on visual information requirements for simulation train- 
ling has lagged for several reasons. The difficulty and cost of perform- 
ing transfer-of -training studies and the incompleteness of knowledge of 
visual perception have been mentioned earlier. Little good theoretical 
work has addressed visual simulation problems. For instance, little 
progress has been made in understanding geometric aspects of information 
</^>\£jit raction from terrain viewed by pilots since Carel's (1961) research. 
Work is limited in this field in part because few CIG simulators are 
available for research. Recently, researchers have used two simulators 
with modern visual systems: the Advanced Simulator for Pilot Training 
(ASPT) at Williams Air Force Base, Ariz., and the Visual Technology Re- 
search Simulator (VTRS) at the Naval Training Equipment Center in Or- 
lando, Fla. Personnel at these' two facilities have made notable efforts 
to deal with some of the research issues described herein, but clearly 
.a much larger enterprise is required to resolve these problems. 

A comprehensive analysis of research needs and priorities in this 
field is beyond the scope of this limited study?; however, some general 
observations may he useful. No one line of research can answer the' . 
questions raised in this report. A mixture of. research approaches is 
needed, ranging from short-term attacks on issues on which operational 
decisions cannot be delayed to long-tetm studies of fundamental issues 
- of information extraction from terrain features. Research is required 
for both visual information requirements and trailing -issues. Much 
attention should be given to designing research methods so that knowl-^ 
edge gained from diverse studies will be additive instead of applicable 
only to the particular simulation and equipment used in each study. \ 

The broad issues of visual simulation requirements need to be 
broken into discrete components that* are potentially solvable. This 
task alone will be no small feat. Current research attempting to de- 
fine relationships between visual simulation variables and flight con- 
trol performance has immediate value for evaluating specific displays 11 
and equipment features, but it will not provide fundamental knowledge/ 
that is cumulative and that might allow prediction of visual informa- 
tion requirements for a wide range of simulation training tasks. Vision 
scientists in academia may be able to help devise improved research 
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approaches. This effort will require, however, long-term commitment of 
these scientists to simulation issues and availability of facilities for 
such work. 

Effective research on visual simulation requires work in both 
laboratory settings and simulator facilities. In both settings, more 
interaction between basic scientists and simulation engineers is needed. 
Most vision scientists in academia are not familiar enough with flight 
simulation problems to be aware of how their research might be applied. 
Simulation studies appear mainly as technical reports, rather than in 
the journals widely read by vision scientists, and are relatively in- 
accessible to the academic community. Furthermore, few meetings are 
held for simulation engineers and vision scientists to exchange infor- 
mation and ideas. The human factors area, in which such interchange is 
^fairly common, might be a good model for interaction of vision science 
and simulation study. ' \ 

To improve and stimulate information exchange between the academic 
and simulation communities, the steering group suggests increasing use 
of three kinds of mechanisms: - 

1. Conferences : The recent Image II conference illustrates the 
value of forums for interaction of vision H scientists and simulation 
engineers.* Direct person-to-person confrontation is the most rapid and 
effective way to become aware of .and understand the pressing issues of 
the moment. (As ar\ aside, state-of-the-art tutorials in relevant areas 
of scfence might be considered for part of these conferences.) Such 
meetings need to be held regularly to ensure the continuing involvement 
of the scientific community. Also, special sessions in flight simula- 
tion could be encouraged at existing meetings ot scientific societies. 
Such sessions have thji potential of reaching broad segments of the vision 
community. The annual meetings of the Human Factors Society have in- 
cluded s\ich sessions profitably, and a session on visual simulation was 
held at the October 1981 meeting of the Optical Society of America. 
Other societies that might be asked to include sessions on simulation 
include IEEE, the Society for Information Display, and the Psychonomics 
Society. National Research Council symposia, such as those organized 
by^fe Committee on Vision on applied visual problems, could also be 

helpful. I 

2. Proceedings and Review Articles : Proceedings serve the obvi- 
ous function of providing information about the content of a meeting 

*to those who were unable to attend. Unfortunately, proceedings of 
simulation conferences have not always been widely available. Publish- 
ing proceedings in a journal is probably the most satisfactory way to 
disseminate this information. Some consideration also should be given 
to preparing annual reviews of vision research relevant to simulation 
(for engineers) and of . simulator display capabilities (for vision , 

researchers). • 

3. Exchanges : ' Although training fellowships exist that allow 
scientists in academia to visit and collaborate at simulator facili- 
ties, the availability of .such fellowships is not generally known in 
the academic community. Greater effort should also be made to provide 
opportunities for simulator personnel in federal agencies to visit or 
study in university laboratories. Two existing mechanisms that should 



* Monroe, E.G. 1981 Image II Conference Proceedings . AFHRL-TR-81-48, 
AD-A110 226. Williams AFB, AZ: Operations Training Division, Air Force 
Human Resources Laboratory, November 1981. 
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be more widely advertised and utilised for simulation research are the 
NRC postdoctoral fel-lowship program and the Intergovernmental Personnel 
Act. The provisions of this act allow for exchange of senior scien- 
tists, which is crucial. Reserve Officers 1 Training Corps (ROTC) pro- 
grams might be bA>adened to attract science graduates for Military re- 
search on simulation. ^ 

The comprehensive research program needed for simulation issues to^ 
be resolved would obviously be expensive and would have to b$> consid- , 
ered in terms df potential costs and benefits. Simulator training af- 
fects both operational readiness and pilot safety. Current investment 
in research on vision and training issues is apparently small in com- 
parison to the projected size of simulator procurement programs (sev- 
eral billion dollars) • 



SOME STRATEGIES FOR RESEARCH 
Introduction 

Appendixes A through F provide examples of research areas and ap- 
proaches that might help elucidate several long-range issues confronted 
in visual simulation. Primary emphasis is given to low-level flight, 
iii which extraction of visual information from terrain features is cru- • 
cial but little understood. Sfyeral strategies are suggested for ex- 
ploring which visual features should be used in low-level flight: sys- 
tematic condensation of opinions, particularly those of pilots (appendix 
A), geometric analysis of terrain information potentially usable (ap- 
pendix B; , and psychophysical analysis of v^sbal processing modalities 
(appendixes C and D) . Appendix E examines* equipment requirements for 
display of whatever visual information is chosen. It also addresses 
characteristics of visual displays that limit the'kind of information C 
that.. can be displayed. . Thus, appdilddCx E pomplements the other appendixes 
and Is applicable to simulation of £ny flight environment . Several of 
these authors have suggested particular lines of research that could be 
followed within their paradigms. 

The next section of this report summarizes the theoretical per- 
spective and working assumptions of these strategies to illustrate 
their power and limitations. An extended example is given to shc^w how 
these strategies might be combined to analyze visual information ta- 
quirements for low-level flight. Th^ complementarity of these research 
strategies is emphasized. \ 



Identifying Potentially Critical Factors by 
Pooling "Expert Opinions" 

The subjective opinions of design engineers, pilots, and visual 
scientists have played a large role in selecting the visual features 
displayed in flight simulators. Many attempts to identify the visual 
factors or cues that are important for a given flight task have begun 
with ^Subjectively composed list of scene parameters (e.g., field of 
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view, spatial resolution, luminance, color, representation of landscape, 
horizon, clouds, shading, known terrain features, shadows, aerial per- 
spective) and perceptual factors (e.g., binocular vision, accommoda- 
tion, .motion, shear) (Brown, 1976; Gibson , 1950 Kraft et al., 1980). 
The many attributes and factors on these lists may be subjected to ex- 
perimental study, but often they are simply rank ordered subjectively 
and incorporated into equipment design without empirical^ evaluation 

This procedure has several flaws. Some pilots may have useful in- 
sight about the visual cues they use in given flight maneuvers (e.g., 
Langewiesche, 1944), but this insight is generally far from adequate^ , 
for characterizing vision information requirements for, display, fur 
thermore, pilots' impressions of the information they use are sometimes 
wrong (see, for example, Waag's <1981) discussion of pilots' evaluations 
of platform motion). Casual >anking is not a powerful method of analyz- 
ing opinions, and it is difficult to combine systematically the opinions 
of a large number of raters in this way. ' 

Although pilots' opinions are inadequate for determining visual 
display requirements, they.may be highly useful in identifying^features 
that can be studied experimentally. More effective analysis of pilot . 
opinions could be achieved with nott-metric scaling techniques. One 
example of these techniques, KYST, is outlined in appendix A, , Tf sum- 
marize KYST, a jury of knowledgeable persons is asked to create a list 
of all the scene attributes they believe apply to a particular flight 
task, in this case low-level flight. Pairs of these attributes ^are 
then "evaluated as to their importance for this flying task.. ^roS^hese 
- ankimts, a multidimensional space is created in which all the attri- 
butes are located in terms of their importance relative to one another. 
This space can then be reduced by factor analysis., with the re suit ^be- 
ing a measure of how many dimensions are needed to "P^^the factors 
that underlie all the attributes (W and Rodivan, 1966; Kruskal and 
Shepard, 1974). Although these' common factors may not be identilied, 
the analysis is important because it will show which factors might be 
rejected and which need to be considered seriously, qualitatively indi- 
cating their common characteristics. , 

Working Assumptions 

If a visual factor is missing from the original list subjected to 
KYST analysis, it will of course not appear in the final a™^ 818 ; 
This absence, however, does not impede relative ranking of the factors 
that are listed; furthermore, additional factors can be added for re- 
iterative analysis. Multidimensional scaling techniques do not require 
that the list of "cues" or factors be exhaustive. So long as at least 
one item on the list of attributes captures a dimension of relevance, 
then that dimension will be identified. The major drawback of this 
technique is that the pair-wise ranking procedure still depends on 
subjective evaluations. Pilots may be. unaware of their use of some 
factors and may make unwarranted assumptions about the importance of 
others. Nevertheless, pilots undoubtedly have much relevant expertise 
that is hard to draw on adequately without some sort of systematic 
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analysis such as KYST. , By suggesting areas for experimental study, 
this kind of analysis complements other research approaches. 



Information Analysis , - + 

The information ^analysis approach attempts to define the visual 
information requirements needed to perform a flight task. "For example, 
during low-level flight, the pilot must judge aTtitude accurately while 
flying over arbitrary terrains. Is it computationally feasible to use 
stereopsis to solve this problem? If not, which sources of information 
could, at least in principle, provide sufficient data for Reliable low- 
level flight? Is it practical to present this information in a CIG 
display? Appendix B gives an example of this approach, showing that 
the pilot must extract three types of information about £he terrain 
from the display: shape, scale, and orientation. The analysis con- 
tinues by illustrating which simple features in a CIG display could 
provide this information and the assumptions the pilot (unconsciou sl y) + 
must make when using these two-dimensional image f&atures>to ifake thrive 
dimen&ibnal (3-D) inferences. ' / 

Working Assumptions * 

Information analysis assumes that the particular forms of 3-D in- 
formation that the pilot extracts from the CIG display. may be .individu- 
ally isolated along with the visual. processes that extract the informa- 
tion. . It "is further assumed that . those processes place constraints on 
the interpretation of the display, and that these cpnstraints may be 
discovered. These* constraints, are provided by, certain geometrical prop 
erties of surfaces and perspective projection. Human vision must make 
assumptions about title real world in. order' to interpret the ambiguous 
visual information that it receives. When such assumptions are wrong 
in some situations, the perception does not correspond to reality. 

The flight simulator generates 3-D images from an internal model 
of a 3 w D environment, and the pilot 's visual system processes these 
images with implicit assumptions about the nature of that environment. 
If the pilot's 3-D perception of the scene is to coincide with that 
which was intended , then it is necessary that the geometric structure 
of the model, and the way it is portrayed by CIG, obey these perceptual 
ssumptions. To give a simplistic but illustrative exdjnple, we aften 
ssume that intersecting straight lines meet at right angles in 3-D, 
provided there is no evidence to the contrary* That assumption helps 
constrain the 3-D interpretation of intersections in an image. But 
if the internal model of a corner of a field or runway is other than a 
right angle (imagine that the runway is a trapezoid), then the viewer 
will be misled. We usually have several independent ways of determin- 
ing the 3-D shape, hence misinterpretation is not a serious problem in 
natural scenes, with all their richness and redundancy. But in the im- 
poverished scenes of a Simulator, it is important to have the model's 
geometry coincide as much as possible with that expected by the visual 
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system. Consequently, we must understand the computational constants 
(geometric and otherwise) that underlie the solution to a vision prob- 
lem in order^o formulate precisely .design rules that govern scene gen- 
eration and lead^to consistent and accurate 3-D interpretations. 



Visual Psyohophysics 

>..• 

Short-Term Experimentation 

*. Multifaceted experimental analyses (e .g! t Westra.et al« , 1981) are 
useful for evaluating the relative effects of identified variables on 
performance in simulators or on transfer of training. These studies 
are particularly helpful in assessing the importance of the differences 
between subjects relative to the simulator variables, but they usually 
offer little insight into the origin of the individual differences. If 
different pilots rely heavily on different informational -j^fpects of^the 
display, then this knowledge may be lost in a factor analysis. To un- 
derstand, the degree to which each source of flight information can be 
utilized by a pilot, long-term psychophysical |tudies must be conducted 



Long-Term Experimentation 

• ' Another distinct methodology examines certain perceptual abilities 
in detail. The goal is to> derive a quantitative understanding of the 
behavior and capabilities of the human visual system. Although much of 
psychophysics has dealt with the resolving power of the visual system 
(spatial and temporal), much recent effort has concentrated on under- 
standing particular ^module's" of the visual process, such as stereopsis, 
texture, motion, and looming. By understanding the capabilities of 
various "modules," their 'utility for a particular flight task can be 

evaluated. . 

Studies in color vision provide a good example of the value of the 
modular approach. More than a century ago, psychophysical experiments 
demonstrated that the human color vision system behaved as if^it ana- ^ 
lyzed spectral Information using only three color filters or channels. 
This property of the eye, subsequently, was exploited in color photography 
and makes possible practical color television (as well as CIG color dis- 
plays) . By coding the signal characteristics to match those of the 
"channels" of the human color system, all the useful information about 
the spectral content of the. scene is delivered economically and effec- 
tively with a tremendous economy in bandwidth; More recent work, in 
vision suggests hints of other such modules (Richards, 1980), some of 
which may be especially Important for flying tasks. Appendixes C and 
D illustrate this apppoach>veinphasizing how simple tests may be <jevel~ 
oped to isolate and "dissect \ particular module and to assess individual 
differences. The example W appendix C compares sensitivities of 
motion-in-depth and size cha^ifes, both of which are potent sources of 
information useful to low- level flight. 
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The crucial assumption that distinguishes a mult if ac tor approach 
from a modular one,. Is that the modular approach assumes that the visual 
ability under study is mediated by a substantially independent visual 
process. In psychophysical terms, a module (or set of channels) pro- 
cesses ^information independently of another set of channels (i.e . , 
movement or size change). Thifif assumption is imposed by methodological 
pragmatics — there' are analytic means fpr studying simple processes of 
few parameters. If there were dependence on -other visual processes, 
the analysis would become intractable . 

A second assumption common to both psychophysical approaches is 
that the visual ability (under study is relevant to the task of flying 
an aircraft. This assumption is made initially, when first studying 
the ability, but later that assumption may or may not be. veti fled. .The 
quantitative understanding gained through psychophysics may in fact* 
demonstrate that the ability is irrelevant to the task of flying. On 
the other Jian^, because of perceptual sensitivity, the ability may pre- 
dict special relevance, which may then be verified in a transfer-of- 
t raining etudy. 

The third assumption, the requirement for approximate functional 
independence of different modules (at least at the early levels of in- 
formation processing), places limitations oa the range of visual abili- 
ties that are amenable to complete psychophysical study using a modular 
approach*. At some atage of visual processing, the interactions, among 
the individual processes become significant, and at that point any modu 
lar analysis by psychophysics becomes in feasible . 

Integrating the Three Approaches: An Example 
Minimal Information • 



One approach to designing a simulator is to analyze the minimal 
information needed to perform the real-flight task.° For example, if 
it is necessary to J^udge altitude at some point during visual flight, 
there must be adequate information about altitude in the visual scene. 
Although the minimal information required for simple flying maneuvers 

8 This example is concerned with flight performance in the simulator. 
As previously discussed, we do not know how closely performance in the 
simulator must follow real-flight demands for low-level flight to ac- 
complish transfer of training. The minimum information approach is 
conservative in the sense that it should provide an adequate visual 
display for learning skills that can be transferred to real flight. 
We do not preclude the possibility of demonstrating transfer of train- 
ing with very simple visual displays, even with low-level flight^ (see 
previous discussion of learning of perceptual and control skills). 
Such empirical demonstrations, however, may be difficult to generalize 
from one simulation to another. 



such as horizontal flight or landing may be obvious , one of the diffi- 
culties with simulating low-level flight is that this minimal scene 
content is not known." Here intuition is not an adequate guide for 
even experienced pilots have difficulty in isolating and formalizing 
what is needed in the (display (Hennessy et al., 1980) A non-metric 
multidimensional scaling analysis could be a useful first step in iden- 
tifying relevant attributes of the environment encountered during low- 
level flight. The outcome of such an analysis of expert opinion should 
suggest the kind of information required by the pilot to fly at low 
altitudes. For example; if an adeqWe representation of the ground 
plane and horizon is suggested as belonging to one important dimension 
of the analysis, then presumably the pilot needs to know his orienta- 
tion with respect to the ground plane for successful low-level flight. 
If "known human object size" or "shadow" appears on another dimension, 
then this result might suggest that it is necessary to provide some 
calibration of the scale of the terrain so that altitude can be judged 

relia We y will assume for the purposes of this illustration that P"limi- 
nary analysis *8 suggested that the visual scene in l^ 1 ^ f "f V 
provides at least three kinds of information: (1) impact point and 
time, (2) altitude, and (3) depression of flight angle with respect to 
the ground . 

■>•-■'■ • 
..... . - . 4 • » " 

Informational Analysis 

• The central problem of low-level flight is to maintain some specif 
•fied height above the ground (or to fly within an altitude band-) . More 
sScifiJlly, the task is to estimate the altitude above-the ground A, 
and flight path angle 6 of the aircraft* at some future time, t and dis- 

y T where the time t is sufficiently long to allow « /""J^*. 
manluve-; to be made. Figure 1 depicts this simple situation ^which 
the aircraft must fly within an altitude band E^ver hilly terrain. If 
the pilot has complete knowledge of the performance of the aircraft 
(e!g!, its Climb or descent rate) , one (an specify/the minimally curved 
flight contour within the band E (dotted line) . / , 

The problem as outlined above is sth^^tmdidable and must be 
broken down into simpler components, ^viewing the goals of low-level 
flight, it is clear that the minimal objective is to avoid hitting the 
ground! Perhaps the next and more difficult step is to specify the 
visual information needed to clear the peaks, without regard to accu- 
rate altitude, finally, the information requirements or a 9P e^if led 
altitude objective for clearance Can be sought, and flight within an 

altitude band can be considered. 

A breakdown Of the problem suggests the following states of 

analysis: ■ 

0^ '. • - 

9 Note that the flight path is not generally the same as the pitch (or 
angle of attack) of the aircraft. 
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NADIR 

FIGURE 1 Depiction of low level flight task. A * minimum altitude above ground; 

0 « f light p t ath angle; <t> ■* pitch angle (attitude) . Cross-hatching indi- 
cates desired flightband; heavy dashed line is minimal flight path. 

i < ■ f ■ 
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.' 1. Qerive the point and time, of impact for the current flight 
trajectory . 

2. Specify the minimum information needed to calculate depression 
angle, of flight. 

3. Specify the information needed to clear peaks.- *\*\ s,.. 

4. Specify the information needed to clear peaks at 200 feet 
+/- 100 feet. 

5 . -^Analyze minimal conditions needed to fly over special terrains 

within an altitude band. ,*•..' 

6. ' Extend item 5 to arbitrary terrains and flight conditions. 

We will now use the derivation of the impact point and time as an 
example of an information analysis. • 
i 

Deriving Point and Time of Impact Deriving the "point and time of 
impact is relatively straightforward, provided that a velocity, field, 
of sufficient spatial and temporal resolution is present. (This is the 
"looming" cue investigated by Regan and Beverley, 1980, and Regan et al . , 
1981.) The impact point is simply the source of the flow field, as. <$> 
pointed out by Gibson (1950) and analyzed by Carel (^961). ™ The time 
of impact t- lmpac( . is given by 

t-„ „ - e/e # . d> 

£ * — impac t 

where. % is the radial flow rate at eccentricity 6 as seen ty the 

observer. 11 * ■ . •, 

To solve equation 1 a*d to determine the source qf the velocity- 
field, texture and contour* information must be displayed at densities 
and rates compatible with the performance of the aircraft and skills of 
the pilot. To explore these visual requirements, we caj examine the ^ , 
information content needed to determine impact time. • • 

Figure 2 plots the flow rate versus retinal eccentricity for vari- 
ous impact times. A log scale has been used to make the^ curves straight 
lines. For example, consider a time to impact of 500 milliseconds (ms) , 
which might roughly represent the pilot's reaction time. Figure 2 shows 
that for t.y40*jm t flow rate changes from 1/4 to 10 degrees/second 

10 Llew>*lyn (1971) and appendix C report, evidence that human observers 
cannot identify the source of the optic flowtfield. However, as Marker 
and Jones (1980) point out, in fly infect he task is not to identify, the 
point (center of expansion) that does not move, but rather to »elfectya 
point and keep it from moving. The observer conceivably may use/cfther 
sources of information to stabilize the^xpansion point and then ex- 
tract, flow rate information. 

U For simplicity of illustration, the speci»^case of approach to an 
object" lying in the (^server's frontal plane is used. For th| more 
general case of approach to a slanted plane, equation 1 would^have to 
include terms for the effects of foreshortening. >j 
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(deg/s) in the central 1/8 to 10 deg (this is within the range of human 
visual sensitivity to motion). If the display is a 1-meter (m) diameter 
CIG with a 30-ms update rate and a 1,000-line interlaced raster at 1 m 
then the fastest motion that can be generated without skipping a raster 
line is 6 deg/s. Thus, to utilize a 500-ms reaction time capability, 
texture contours would have to occur within 3 deg eccentricity One 
might want to provide perhaps ten times this contour density <*•••. 
3 cycles/deg) in the visual display to allow effective estimation of 
flow rate anV angle. Without this display capability, the simulator 
will not provide all the useful information offered by equation 1. 
"see appendix E for other design considerations.) (As an aside we 
note that this stringent display- requirement is needed only at the cen 
ter of the display. Since the peripheral display requirement is much 
less severe, peripheral edges might be traded for centra ones in some 
graded manner according to figure 2 and human abilities,)^ This alloca 
lion 0 f "center" of course would depend on where the pilot looks'. 

The above is an example of how an information-theoretic analysis 
aimed at designing a display to help the pilot avoi J^J^^Jf 0 - 
ceed If- sufficient information about the time of impact is not dis 
Played the pilot might adopt a conservative strategy, such as always 
keening the flight path above the terrain, but this strategy Preludes 
controlled terrSn following. Is this conservative behavior elicited 
by the inadequacies of the simulator, the best training for the P^ot? 
It It is not then the capabilities of the simulator should be upgraded 
to provident least the minimum information content needed to compute 
impact point and time. 

T^^o.^n wHed to Clear Peaks 12 To hint at possible solutions 
to t he increasingly difficult stages of the problem, we briefly con- 
sider some additional information for clearing peaks. 

Note that if the sky is textureless, then the source of the flow 
field must be inferred from the flow pattern of the terrain contours 
Once again therefore, an analysis of the display requirements should 
S Ld! to ensure that tL terrain content does indeed provide suffi- 
cient spatial resolufiorJand detail to allow such an extrapolation to 

1,6 Without such information, a pilot is forced to look for other cues 

Again! an information-based analysis similar to that described earlier 

. 12 For suggestions regarding the extraction of altitude, see Harker and 
Jones (1980). 

13 Note that the alignment of the aircraft with a near ridge and the next 
peak is a good strategy for ensuring that the flight path angle is nega- 
tive with respect to the ground plane. 



can 'specify the minimal display conditions , given the aircraft perfor- 
mance and the reaction times and visual abilities of the pilot. 

Terrain Following Finally, if thg full mission is to fly in- the 
band 15 of figure 1 as originally proposed, then clearly the pilot must* 
be able to perceive terrain shape fairly accurately. At present , it 
appears thafc the information required to \lo this is beyond reasonable 
technical capability, given the demands of low-level flight and the 
present optical and computational limitations of the available graphics 
displays. However, a computational analysis could profitably explore t 
the possibility of a sufficiently rich display region for the lower 
portion of the display. Such a region might include only a small frac- 
tion of the total display field, allowing the possibility of at least 
an order of magnitude increase in texture content in a critical visual 
sector with little loss elsewhere. (See appendix D for other field-of- 
view considerations.) * 

Summary of the Approach The design pf an optimally efficient simu- 
lator display requires an information analysis of the task that keeps 
in- mind the performance of the aircraft, the visual abilities of the 
pilot, and the display capabilities. The formal steps in such ah analy- 
sis are as follows: 1 

1. A clear statement of the ultimate objective (task) . 
-2. The subdivision of this objective into the component problems, 
from the simplest to the toost complex. 

3. An analysis of the minimal information requirements needed to - 4 
solve the simplest problem. 

4. An extension of this analysis to the more complex cases, show- 
ing how the required variables can be extracted from the dis- 
played scene . • 

5. Consideration of the display and human capabilities that limit 
the available ranges of the variables of interest. 

Applying Psychophysics 

The non-metric scaling technique is designed to suggest which as- « 
pects of the environment are important *to a task such as low-level 
flight. The information analysis then shows what must be included in 
a two-dimensional visual display of the simulator to allow a pilot to 
extract this information. However , because the results of each approach 
depend on certain assumptions, it is necessary to demonstrate ttjat the 
observer does indeed have the ability to sense this information with 
sufficient accuracy. For example, a high-resolution binocular visual 
system cani-4n principle, recover a complete 3-D terrain map and hence 
reveal distances, slopes, altitudes, and velocities,. The threshold for 
human stereoacuity , however, limits the range of 'binocular vision to 
distances" that would be. covered by a^high-speed aircraft in seconds. 
A critical component of the analysis of a problem tp-be simulated is 
to ensure that the displayed information is not only relevant but use- 
fill. At the same time, psychophysical studies of the parameters of 
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interest have the potential of foretelling whether one individual will 
pe/form -better than another on a well-defined and understood task _ . 
(Regan et al., 1981). We will illustrate these points using the sim- 
plified low-level task of deriving the time for impact. (For addi- 
tional experimental suggestions regarding the position of impact (or 
flight direction), see appendix D.) •/. 

Time to Impact Time-to-impact information is contained in the 
optic al flow field, but can the human observer use this information? 
If so- is all the information processing capability of the observer 
utilized inThe simulator? A simple experiment by Carel (1961 (sug- 
gested that time to impact can indeed be estimated as P^f fe ™ m 
equation 1, without knowledge of the distance to the surface , object 
size or speed. In this laboratory study, with an expanding flow field 
of random dots, subjects could easily make extrapolations to the time 
to impact when'the field expansion was stopped be ore simulated impact 
(see figure 3.) .14 The range of judgments is substantial, however. Is 
this ange due to a limitation of the apparatus or laboratory setting, 
or is Ti an inherent inability of the human visual system? To answer 
tnis question requires further psychophysical study that fits nicely 
into the module or "channel" approach. 

s< Ch ange Channels deferring again to figure 2, we note thaV 
the c ritical information a bout time to impact is conveyed by a change 
in anguJar size at a given eccentricity. To solve equation 1, we must 
either be able to measure both the rate of change in angular^size 6 , 

a tic rpHnal eccentricity, 6, and then take the ratio or measure 
Sfi airecav For illustration we will assume that the human observer 
can reliably'recover 6 and examine the psychophysical constraints im- 
posed on displaying and using the derivative 6, recognizing that the 
more useful parameter 15 might be 6/6. nt ,. , . 

Suman capability of measuring size change (6) has been studied by 
Regan Z ^Beverley (1979) . Figure A sh ows threshold response data that 
ReLn and Beverley discuss in terms of tuning characteristics of a 
size-change channel. Optimal sensitivity for all three observers is 
near a rale of 3 Hz for 1/4 mih arc, corresponding to a velocity of 
1,11 deg/s! (A more conservative threshold rate estimate for most of 

1A This does not imply that the observer necessarily could identify the 

4 t f ^l«rt from the flow field (see footnote 10). Furthermore, 
a : fgu us?nd C :rrorus e perception of expanding patterns U , pos si be 
as illustrated by the Ames trapezoid illusion. Ever i with full °P£^_ 
flow patterns, what is perceived may depend on the state of ^e observ 
ot n«rticularlv when the scene content is impoverished in other re 
: eX^rtner experimentation is needed^ relate these laboratory 
data to visual simulation (e.g., Harker and Jones, 1980). 

15 To test whether 'the human observer uses or 6/6, one can compar e the 
combined variance for estimating 6 and 6 separately with that for esti 
mating -th* ratio 6/6. 
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FIGURE 3 Carei's (1961) data for the reliability of, 
estimating Impact time. 
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FIGURE A Tuning curves for size-change channels 
(adapted from Regan and Beverley, 1979). 
Dotted line - changing-size stimulation 
producing a size-change sensation; solid 
line * changing-size stimulation producing 
a motion-in-depth stimulation. 
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the frequency range would be 1/10 deg/s.) If the full capability of 
the human observer is to be utilized to extract impact time from rate 
change, then both the raster size dntflrate must meet these limits. 
For a 450 raster-lirip display at 1 m, the raster would be 10 lines per 
degree per meter of display. This translates into 6 min between raster 
lines for aim display, so l/25th m display diameter would be required 
if the full capability of the human observer is to be used to solve 
for time of impact. 

Similar psychophysical constraints can be determined to set an 
upper bound on the detectable change in size, using the readily avail- 
able spatial-temporal contrast sensitivity functions' (Kelly, 1979; 
Wilson and Giese, 1977). Since these functions change with retinal 
eccentricity, so will the limits for the useful flow fates. 

Figure 5 illustrates how human capabilities might restrict the 
range of flow rates that are useful for calculating impact time. The 
minimum spatial resolution for the human observer is also included to 
show how the available flow rates are restricted to a portipn of the en- 
tire range. The useful region is, of course, based on normative data. 
Regan and coworkers (1981) have shown that on related tasks, individual 
differences may run as high as 80 to 1, suggesting a wide range of 
abilities for successful crash-avoidance during low-level flight (see 
appendix C) . These large individual differences may well be responsi- 
ble for the wide range in Carel f s (1961) data previously discussed 
(figure 3). However, further work is needed to establish the role of 
size-change mechanisms and to determine whether they are critical for 
low-level flight. 

Evaluating "Optic Flow 11 The expanding pattern created by forward 
locomotion includes a plethora of information about the world and one's 
•motion in that world. The problem is to determine how the retinal pat- 
tern can be decoded to^etover the aspects of the world that are rele- 
vant to the tasks at hand. Although some progress has been made in 
this area (Longuet-Hlggins and frrazdny, 1980; Oilman*. J&J9) , still more 
theoretical and psychophysical work is needed. The brief information 
analysis in the preceding section suggests that the calculation of the 
flight angle from the flow pattern would be the next problem to tackle 
after the derivation of impact time. To solve this probldm, the ground 
plane must he estimated from the retinal flow pattern. Appendix D dis- 
cusses some aspects of this problem. A study by Harker and Jones (1980) 
is also relevant. Rather than exploring various parameters of the 
retinal flow field solely with psychophysics, however, we suggest com- 
bining research strategies, as illustrated in this report. 

f 

Evaluating Vision Research 

The integrated research strategy illustrated above should provide 
a powerful approach to the fundamental problems of developing more ef- 
fective visual displays for simulation. This strategy suffers, however, 
from the "divide and conquer 1 * approach typically used in the laboratory 
to isolate and study individual processes. This approach fails to 
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FIGURE 5 Restrictions^ placed on the useful impact-time loci of figure 2 because of 
the sensory limitations of the human visual system. 
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examine visual perception in its natural state, in which visual input 
is usually rich and redundant. Vision isj not merely a collection of 
independent processes. Sensory information from the ey^s is integrated 
in a complex manner with knowledge of £he- world and expec tations of what 
will be seen, which are continuously modified as the observer interacts 
with the environment. Thus, vision is intimately tied to sensorimoti 
behavior and intellectual problem solving. Studies of the individual 
processes that derive information from the teyes do not shed light on 
these interaction^. The human ability to usevarious sources of infor- 
mation and a range of strategies to perform reH^World tasks suggests 
that mote than one approach to visual simulation of^a given flight task 
may be effective. It is not possible lo determine the most effective 
approach to visual simulation solely from, considerationBsOf vision. 
All of this indicates that suggestions arising from this bx any other 
research strategy will to be empirically evaluated to determine 

their value for simulation flight training. 
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APPENDIX A . 
' ANALYSIS OF NON-METRIC DATA 
7 D. H. Fender 



It seems almost a truism that pilots must know a great deal about 
those aspects of the visual environment that contribute to the success- 
ful performance of their task. Simulation researchers, however, report 
that the subjective assessments of pilots are not a reliable guide to 

-the visual ^information actually used ii£,f lying. 

v rf y evaluation of the problem is ffcat pilots are probably not con- 
sciously aware of what they know about this problem, cannot turn this 
knowledge into reliable verbal forms ,^and , that the information is prob- 
ably non-metric anyway. I persist in my belief that there must be a 
mine of information residing with pilots; the major problem is to de- 
vise reliable ways of tapping and analyzing this information. ? 

There are two problems here— how to tap. the information and how to, 
analyze it. Let me deal witjk, the second problem first. Since the data 
are most t*M.y non-metric, tne analytic method of my choice would be 
multidimensional scaling with an unknown proximity function, a6' sug- 
gested by Shepard (1962). This analytic method requires.-that we haye a 
confusion matrix (or prptfimity matrix) .between the nonparametric prop- 
erties of the simulator visual environment that we wish to study, and,, 
this dictates the form of experiment that we should »do to tap the pi- 
lot's experience. 

For example, if we wish to. analyze the present simulator we might 
start by asking the subject, to write down a list o^all the attributes 
they can think of that apply to the ^sual. environment of the simulator 

. We would probably get a pretty conVenVtonal- list, such as 

(a) field, of view 

(b) monochromatic display ' > • 
, (c) collimated viewing condition 

(d) resolution of the display 
*> (e) representation of landscape . . 

X (f) movement of display contingent on maneuvers of plane, 

and a few unusual items, such as 

(g) fingerprints on cockpit canopy 

(h) flicker in peripheral vision ' ' 
Some might even object to the projective geometry used in the display, 

and record ' 

(i) Euclidean geometry! 

and so on . 
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. tie. would then take the combined lists of all of the subjects, pos 
sibly add a few attributes of our ottn> and then put the following prob 
lem to the subjects: 

In the design of a future simulator we shall not be able to in- 
corporate all the attributes of the one with which you are 
familiar. You will therefore be asked to select between pairs 
of attributes. Xour task is to say which of the pair you -would 
retain in the new d&eign. We malte the assumption that t>he at- 
tribute you do not select will be (i) eliminated (as in the 
case of platform motion), (ii) degraded so as to be/ useless in 
trhe performance of the task (as in the case of^resolution) or % 
(iii) changed so that the effect on you becomes* objectionable 
(as in the .case of peripheral flicker). In short, a vote 
against an attribute is always tx> be interpreted as "change the 
attribute so as to degrade the pjerformance of the flying task. 0 
Thus a vote against fingerprint^ does not mean "remove them/ 1 ; 
it means "make them worse." "Soto^pairS may seem to -represent 
an impossIT^Te choice, for example, contrast versus tesolution, 
.since complete degradation .of either will kill the» visual in- . 
put; but plpase make a choice anyway! 
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A consideration of these numbers alone* begins to give some idea of 
how the subjects perceive the flying task; for example, twenty voted to 
exclude (c) in relation to (a), but equally twenty voted to include (c) 
when compared with (d) . Already we begin to see the relative merits of 
(a), (c) , and (d) insofar as the subjects are concerned; we coul^ pos- 
sibly lay them out along a decision axis, such as that shown below: 



(a) 



20 



--*(C)«" 



20 



Keep 



(d) 



Decision 
-♦parameter 



Reject 



But this is patently wrong when we consider (d) against (a) — nineteen 
voted to keep (d) and reject (a)! It is probable that the (a,d) de- 
cision was made on a different parameter from that used iri the (a,c) 
decision or the (c,d) decision. In this case, a better layout might be 
as follows: 



Keep 



Decision 
parameter 2 



Reject 
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Decision 
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Obviously, the geometry doesn f t quite work out, but it might be possi- 
ble to juggle the layout so that at least the rank ordering of the in- 
terpoint distances is monotottically related to the experimental data. 
We might be able to do this in our heads for a small number .of data 
points, but. the complexity increases rapidly with the number of points. 

Fat a group of N points, any prescribed rank order ofr>the N(N-l)/2 
interpoint distances (including ties) is realizable in an Euclidean 
metric space provided it is of dimension N-l or greater. Suppose that 
we had tested nineteen attributes;* this means we could display the 171 
proximity measures as metric distances among nineteen points in an 18- 
dimensional space. Apart from the obvious problem of visualizing an 
18-dimensional space, such a configuration would yield little more in- 
formation than, the simple rank ordering of the 171 proximities. It is 
therefore highly desirable to reduce the order of our metric space if 
we can do so without distorting the representation of our proximity 
measures. 

The mechanism for reduction of dimensionality proposed by Kruskal 
(1964) is based on finding a structure that maintains the monotonicity 



condition. ,For a given configuration in a space of dimension n, where 
n ^AN^lj a monotone regression of distance upon proximity measure is 
performed an^ the residual variance used as a measure of how well the 
particular configuration satisfies the mdnotonicity condition/ This 
measure Kjruskal calls th? stress* the conf iguration is changed itera- M 
tively following a nonlinear minimization algorithm based on the method 
of steepest descent until the configuration with the smallest possible 
stress is obtained for that order space. Hence, the dimensionality of 
the space may be reduced so long as a configuration "can be found that 
yields a value of stress that is acceptably small. If this condition 
is satisfied, then the ^structure in the lower-ortl€r space is just as 
valid a representation of the proximities, as the 18-dimensional struc- 
ture. A large stress value implies monotonicity is not maintained and 
hence the configuration does not very well represent the psychophysical 
data. p " * » "? g 

It is of considerable interest to determine the minimum dimferi- 
sionalify space in which a configuration can be found that adequately 
represents the psychological proximities. : The number of dimensions \ 
required can* then be associated with the njumber of fundamental proper* 
ties of the stiiquli upon which the discriminations are based. But it 
is also important to remember that this scaling technique is only a 
conceptual convenience; visualization of a configuration in a low-order 
space is facilitated , and can repeal important aspects 1 of the^ interpoint 
relations. 

I have u^ed multidimensional ^caling/in a number of contexts, and 
my experience has been that the human can make decisions only in a space 
of very low dimensionality. Typically, the^urves of stress versus di- 
mensionality in an experiment designed to differentiate between visual 
textures (Santoro and Fender, 1976) look ffhul: 




Curves such as these indicate that a 3-dimensional configuration 
would probably. describe the data adequately, and 'a. model of higher di 
raensionality would overfit the data. 
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It is at this point that the wit of the experimenter enters the 
problem. The computer merely churns but a mass , of numbers; the nine- 
teen, attributes are positioned in n-space so as to minimize the stress, 
where n is chosen by the experimenter from a curve >such as the one 
above • The computer gives the coordinates df the points representing 
the attributes. For simplicity of visualization, let us ass^je that we 
can represent the ^data in 3-space. We next perform a varimaf^rotatiori, 
that is, we find the axis sudh that if the data pbints were projected 
onto a plane perpendicular to this axis, the^ variance of the data would 
be reduced by the maximum amounts The two other orthogonal axes satis- 
fying the same condition are also calculated. We then. have to think. 
Suppose that the data were as follows (reduced to 2-space for further 
ease of visualization): 




That is, the data points lie very neatly along the varimax axes AA and 
BB. We obviously have two intermingled classes of <feta, one differen- 
tiated by whatever property is mapped aJ.ong.AA and the other by the 
property mapped along BB. the experimenter then has to examine the 
attributes that are mapped along AA; only his insight will help him 
understand what property lies -aleH&^Vand is scaled such that the at- 
tributes at one end (the analysis dokeA't tell you which end!) have "a^ 
lot" Of this property while those at the other end have "not very much" 
of the same property. The sstoie process can be performed along BB. The 
analytical thought process becomes more complex if the data points are 
as shown : 

A .B 

■ \ • / 



That is, there are many points Such as C that do not lie on AA or BB. 

The point C can have any lot/little combination of the properties AA and 
BB, since we do not know a priori which way the properties are scaled 
along the axes. In general, one uses the points lying on axis to de- 
velop ideas about the property mapped along each axis, and then one 
checks the off-a^ispoittts against these theories. Of course, it may 
happen that no points in the data structure lie on axis! 

Th^outcome of this analysis'— which i« not exact by any means — is 
a stronj indication of the parameters the subjects used when making 
their decisions to accept or reject an attribute. Whether these de- 
cisions have any link with flying a Simulator depends on the skill with 
which tie original experiment was set up. But suppose that the work is 
well doiie; the design principle is then self-evident—add (or enhance) 
features whose attributes lie at the "good 11 end of the longest axis, 
and if compromises have to be made, lop off features whose attributes 
lie at the "bad 11 end of any axis. . 

^ The most evident criticism of this approach is that if an attri- 
bute is not included in the original list, it will not appear in the 
final analysis. This is a potent criticism but fortunately not a fatal 
one—the analysis proceeds perfectly well without the missing attribute, 
and in many cases an examination of the, axes of the final structure 
prompts the experimenter in regard to attributes that may have been^ 
missed. The wo^k cart then be repeated in an iterative fashion. 

The experiment described earlier is put forward only as an example 
oL^how multidimensional scaling might be applied. Many other experi- 
mental arrangements are possible, and workers more closely in touqjh ~ 
with me simulator problem are surely the ones to design the actual ex- 
periment. For example, rather than giving a colloquial explanation of 
what a) vote against a particular attribute might mean, it -may be better 
to §^at the subject in the simulator and say, "If you vote against 
resolution, this is what will happen," then tweak the knob to blur the 
imags, and so On over all the attributes, before the subject is asked 

» to cast his vote. 

* Further, instead of adjudicating between attributes in a simula- 
tor, the subject might be asked to evaluate an attribute in the simu- 
lator against the same attribute in a real plane, or in another model 
of simulator, on a scale of one 1 to five, for example* 

A program to perform this analysis is available commercially. It has 

;many features not treated he re- They are described in Kruskal e t al. 
(1973) . One feat Mre, however, is worth mentioningV The program has the 
ability to analyze data represented by ,two or more proximity ^matrices 
Simultaneously. That is, it derives ■ the structure that is^ the best fit 
to both matrices. One way to exploit this would be /as follows: If any 
objective data exist on the merits, or relative merits, of any of the 
attributes, these could be plugged into a second irfatrix. It is not 
necessary to fill" all the ce 5 lls of the matrix. The program works quite 
well on an incomplete matrix. If the analysis Is then run on the two 
matrices, using varying weighting factors between them, at the least, it 
1g possible to test the concordance of the suj/lective and of the 
objective data. 
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In summary, I propose that the method of multidimensional scaling 
provides a technique that could be applied rapidly % and at lpw cost to 
the backlog of objective information concerning flight simulators* and 
to subjective information that could be gathered without too much ex- 
perimental effort. At worst, the analysis might tell us no more than 
we already know, but at best it might give considerable insight into 
the functional attributes of a flight simulator and of the interactions 
between them. 

Other aspects of multidimensional scaling are explored in Shepard 
(1974), Shepard et al. (1972) , Kruskal an* Shepard (1974) , and Egan 
(1971). 
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APPENDIX B 

COMPUTATIONAL ANALYSIS: IMPLICATIONS FOR 
VISUAL SIMULATIONS OF TERRAIN 1 

Kent A." Stevens 



Introduction, 

Low-level flight (LLF) is a flight regime of increasing importance 
in- combat and also a regime where significant difficulty has been ex- 
perienced In its simulation. Pilots who are expert in actual LLF often 
cannot duplicate their performance in the simulator. When "on the deck 11 
in \he simulator they have difficulty both in following the. contour of 
the terrain at low altitude (without crashing) and in judging altitude. 
In part, the- difficulty is technological; for instance, the very rapid 
motion of terrain detail across the display require*} fast display pro- 
cessors. Other difficulties ate perceptual: the visual displays must 
convey an adequate 3-D understanding of the* terrain over which the pi- 
lot must' fly. The combination of technological and perceptual problems 
has prompted some reexamination of the methods for terrain depiction. 
This t paper describes a new technique that may improve the visual simu- 
lation of terrain, as applied to the simulation of LLF. 

The introduction will discuss three background issues. The first 
concerns flight training. Research and development efforts regarding 
simulators for flight training should be evaluated in terms of their 
impact/ oil training. However, LLF simulation presents some basic visual 
problems that must be solved regardless of whether the simulator is 
used for training or any other purpose. Hence we concentrate here 
solely on the visual problems of LLF simulation. Next, we show that 
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although real world scenes are distorted ^computer image generation 
tciG) displays; they can.still convey useful 3-D information. With 
that observation we turn to problems of defining "information con- 
tent ° Finally, we discuss a computational methodology for studying 
num^ vision that approaches complex systems at several level* 'rf^T 
tail or specificity, where different theoretical tools are appropriate 
at each' level / This approach has led to succinct and precise d«crlp- 
fions of various processes of human vision (such as stereopsis and^edge 
det^tJon, S>T^fr). That point is crucial for its Wggjj^^ 
flight simulation .^ven though we currently have only a limited under 
standing of the oSkall visual system, many facts that we do know 
translate into important suggestions for simulator scene generation. 
The reet of the paper attempts to show how that insight comes about. 

Some Remarks Regarding Training . 

The flight simulator that ]we have in mind is a device for tracing 
low.level flight. One school ft thought is that a t raining devic^ 
sCld be optimized for the particular training task for which it is 
intended! Consequently, the distinction between training a skill to 
'a novice and training for performance or skill maintenance should^e 
reflected by the simulator system, for example. But it 18 / oub " u * ™ e 
state of the art of'LLF simulation is sufficiently advanced as to make 
tnls distinction worthwhile^ of yet. There are clearly two rather . 
aiverse types of research required: applied visual "ience^order 
!« ^rnvp- the effectiveness of the visual display, and training re- 

Sri s ^^i^^^^hr 

Ing, or overlook problems that are important to training. That danger 
exists but the problem* that beset LLF simulation are probably in- 
tractable unless some decomposition is made of the overall complex of 

iS8Ue The pragmatic view taken here 'is that careful ^^^^ 
veneration of visual scenes (that leads to greater precision Jn^er 

optimization 01 "« necessarv to understand how the 

versus skill maintenance, say), it is necessary to u» reflect 
two training tasks differ and how the visual displays should "fleet 
those d ferences, in principle. But these issue*, ■"£"v"£ C E«t- 
For example even in the case of simulated landings (probably the best 
ItudieTf Ught maneuver for simulation) , it is neither intuit ve no 

o my knowLdge) empirically known hpx tp opt im^ the visual display 
for maintenance versus undergraduate training. Thus there are two - 
issues at hand: visual perception and optimal pilot training. Effort 
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on one problem shoulcj not be madje at the exclusion of the other. How- 
ever , we will not deal with issues of training here. \ 

Difference^Between CIG Simulation and Real Terrain 

When the CIG simulation of LLF is compared to the real world seen 
from a low-flying aircraft, the simulation seems oversimplified and \ 
somewhat corrupted by artifacts, such as the stuttering. effect that 
results when the/CIG system cannot update the display in the periphery 
often enough gjtfen the large angular velocities associated with LLF, 
One might conclude that the 3-D impression would be adequate if only 
one had greater display capability. But it is conceivable that even a* ■ 
hundredfold improvement would not delve the problem. 

Insufficiency of .detail is not the whole'story. A simulation scene 
is not merely a simplification of a real scene; there are dramatic and 
qualitative differences. In reducing the bandwidth of a real visual 
scene to manageable proportions, there is no straightforward sampling 
technique available~we cannot merely take every n-th bit of informa- 
tion, as it werg. Instead, we construct a novel and unique world out 
of relatively few edges and perhaps shading, color, points, and lines. 

The visual world we construct in the- simulator shared many simi- 
larities with the rkal world. The single most important Similarity is 
that the-rules of perspective geometry are preserved in the simulation. 
That means the visual world in the simulation behaves optically as we 
expect. This pqint should not be underrated, for it is responsible 
for muchfof the success of visual simulat-ion. 2 Specifically, the per- 
spective transformation by which the- simulator CIG system projects 3-D 
points, lines, and surfaces from the terrain model 6nto the 2-D display 
screen (and in .turn, onto the retina so long as the( display is viewed 
from the proper location) precisely corresponds to the way real 3-D 
points project onto the retina. 3. Consequently, any static view is 
reproduced with geometric precision (e.g., the proper "texture gradi- 
ent" is generated because increasingly distant surface elements appear 
progressively smaller) . Also, if the surfaces are modeled as opaque, 
the CIG system (usually) has the capability to make nearer surfaces 
occlude from view those that are farther. Moreover, if we move in 
.space, the continuous chapges in perspective of points and surfaces 
also correspond gsomdtrixally to what happens in the real world. Con- 
sequently, we also have geometrically correct "optic flow," "motion 
parallax," and so forth (see later). It is the fact that perspective 
projection is accurately duplicated by the CIG system that was 



Here we are suggesting that much of the three-dimensionality comes by 
having correct perspective, regardless of what is displayed in perspec- 
tive. But that is not to say that some benefits cannot be achieved by 
carefully crafted distortions to the perspective (see Finch, 1977). 

3 

But see Kraft et al. (1980) for engineering considerations. 
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responsible for the compelling effect of space In early night-flight 
simulators, where only moving luminous dots (such as runway lights and 
city lights) were projected. As opaque surface shading was added and 
daytime scenes were simulated, the additional gains came, in part, from 
the visually apparent occlusion of distant objects bv nearer ones. But 
along with attempts to simulate daytime scenes came the realization 
that the simulation is very different from the real thing. 

One difference of probably minor importance is that the display is 
effectively a monocular presentation (see later). But just to note a 
few of the profound differences between the real scene and the simu- 
lated counterpart* observe that real surfaces have detail at all scales, 
and that new detail is continuously revealed as one approaches a sur- 
face. But the limited detail that most GIG systems can display com- 
bined with their resolution limits result in drastically simplified 
scenes. (This fact may be particularly relevant to the LLF question, 
as pilots in real flight conceivably use fine recognizable detail of 
vegetation and rooks in order to judge their distance from approaching 
terrain.) Another difference is that real surfaces are not restricted 
in their 3-D- shape (consider a rolling desert floor or an eroded can- 
yon wall) . But the usual computer representation of curved surfaces 
in terms of planes bounded by polygons results in a dramatically dif- 
ferent sort of terrain, than would be found in nature. As it is not 
simply the fact that the surface is piecewise planar rather than con- 
tinuously curved, It is also not true that sharp straight boundaries 
occur at every junction. Those straight boundaries are also used to 
denote field boundaries (such as the checkerboards which resemble agri- 
cultural land). In fact, the basic element in simulation for represent- 
ing surfaces Is an edge across which screen intensity Is sharply dis- 
continuous. The intensity edge may represent a sharp physical feature 
such as a cliff, ridge, or comer of a building or a place where the 
surface reflectance sharply changes, such as the border of a 'runway 
or the edge of a checkerboard square. Some display systems have shad- 
ing and modulation of contrast to approximate atmospheric haze or fog, 
but generally the Intensity edges are equally sharp and homogeneous in 
CIG simulations, and that is not natural. Another point is that actual 
Surfaces have physical texture in relief above the mean surface level 
(bushes above the ground, peaks and troughs of waves) but this is oostly 
to generate in current- simulator systems. One final difference should 
be pointed out: actual surfaces reflect light in a complex way depend- 
ing upon the orientation of the light source (the sun and the overall 
sky illumination),- the orientation of the viewer relative to the sur- 
face, and the physical properties of the surface. But generally the 
intensities of simulated surfaces are unnaturally constant and homo- 
geneous—even- the most sophisticated techniques for shading surfaces 
are highly simplified. ' 

This discussionas not intended to point out the well-appreciated 
fact that CIG scenes are unrealistic. Its purpose is to show that 
simulated terrain ifran extreme and stylized simplification of the real 
wdrld. That is not necessarily detrimental, however, for human vision 
has a remarkable ability to ignore simplifications in illustrations. 
For instance, we are all familiar with the strong visual impact that 
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even simple line drawings provide, such as seen in engineering and 
mathematicatexts and in' assembly instructions. While they are highly 
unrealisticT^e seldom attend to that fact. Since they carry the 4 
necessary 3-D information, they, serve their purpose; it does not mat- 
ter that actual surfaces give rise to more complicated images. Simi- 
larly, it may be argued. that simulation displays of terrain may be 
adequate (for purposes of training, etc.) and yet highly unrealistic 
and caricatured* 

Problems with Intuitive and Geometric Argument * 

Thus we have that the CIG display tn^y carry the necessary 3-D in- 
formation despite the unrealistic qualities just described. But we 
still have to formalize what we mean by ,l hecessary 3*-D information. 11 
In that pursuit it should be stressed that our intuition should not be 
trusted. Intuition is often wrong regarding the way the» human visual 
system operates. In short, what we naively believe to govern our 
visual perceptions is often not the case. It is worthwhile examining 
one such example in detail. The example involves the familiar (static) 
texture gradient, specifically, that the texture density seems intui- 
tively to be the crucial depth cue in texture gradients the higher the 
density, the greater the distance to the surface (Gibson, 1950); see 
also the geometric analysis in Purdy (1960) . In fact, distance cannot 
be inferred directly from density (the following argument is summarized 
from Stevens [1980]). The reason is that texture density is a function 
not only of the distance to the surface but also of the slant of the 
surface relative to the viewer (the greater the slant, the greater the 
foreshortening and hence the greater the texture density). In the case 
of an arbitrary surface, one cannot decouple the relative contributions 
to the density gradient caused by foreshortening from that due to dis- 
tance. Consequently, one cannot infer distance from texture density. 
This argument provides an explanation for the largely ignored psycho- 
physical evidence (e.g., Smith and Smith, 1957; Braunstein, 1968; 
Braunstein and Payne, 1969) that^texture density is an ineffective cue 
to distance. This result is contrary to our intuitions. It points 
out that we must go beyond introspection. 

In the same vein, it is not sufficient merely to examine the geo- 
metric properties of perspective projection. For instance, the analy- 
sis of texture gradients that Purdy (1960) performed set out to find a 
geometric basis for the hypotheses that Gibson (1950) set forth (such 
as the one just discussed). But such geometric analyses must be care- 
fully regarded, as they implicitly embody certain geometric assump- 
tions, such as that the ground is globally planar (see Purdy, I960). 
The mathematical relations derived with those assumptions, of course, 
do not hold when the physical surfaces are not so constrained (e.g., 
when the textured surface is not planar). It is not a straightforward 
matter to set down the geometric relations that provide the basis for 
visual interpretation of the image. The relations necessarily incor- 
porate constraints, and it is a separate and nontrlvi^l matter to 
verify that those constraints are adopted by the actmal perceptual 
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processes of human vision. For further discussion on the matter of 
constraints and its relation to Gibson's "direct perception" hypothe- 
sis, see Ullman (1980). 



How Is Information Encoded in the Image? 

~Image information content is usually discussed in terms of depth 
cues such as texture gradients, optic flow, and stereopsis. But to be 
useful, in particular to be useful for flight simulation, those terms 
must be defined more precisely than is usual in the psychological lit- 
erature (see also Hennessy et al., 1980; Semple et al., 1980). To 
illustrate, consider the difficulty a simulator designer would have 
trying to, apply a general fact about optic flow. For argument, sup- 
pose he reads that optic flow provides information about the direction 
of travel because of the streaming of detail radially away from the 
point he is approaching (Gibson, 1950). If the designer learns that 
pilots often lose vis ual\ orientation in the simulator (while maneuver- 
ing, say) , the designer would recall \hat optic flow is. relevant . But- 
the designer is now. stuck, in a word, because optic flow was not de- 
fined with sufficient specificity that the designer might translate 
that suggestion into an improved visual display. It is a rather, subtle 
point that optic flow is inevitable whenever one moves through other 
than empty space. Optic flow is a consequence of perspective projection 
and motion, so regardless of how the terrain is depicted, if one moves ^ 
across.it, the resulting movement of detail across the visual field is 

opttc flow. P * 

The designel^is faced with a terrain simulation that is apparently 
inadequate, seeks tV improve it, and knows that optic flow is relevant. 
So\e considers the Variables that govern'optic flow, those being the 
parameters of motion relative to the surface as well as the qualities 
of the surface texture itself. (The faster and lower the aircraft, the 
more pronounced the apparent optic flow; also, the more densely tex- 
tured the surface, the greater the effect.) Of these variables, it is 
only the surface texture that the designer may manipulate, but how to 
do so? Insight would come from knowing something about how the human 
visual system measures and internally encodes moving texture and how it 
extracts 3-D information such as orientation from those measurements. 
We cannot dismiss this issue in a cavalier manner, trusting that the 
human visual system will extract what it needs so long as the CIG dis- 
play is rich enough in detail. Current (and foreseeable) CIG display 
technology does not (and probably will not) have the capability of 
reproducing the complex visual text,ure presented by real LLF over 
natural terrain. Rather, we should devise more efficient uses of the 
CIG system's ability to generate texture. We might think of this as 
"impedance matching," of matching the type of texture that U displayed 
with the type of visual processing we make on that texture, s\> as to 
optimize the information transfer. For instance, the visual display 
might concentrate its detail at the (instantaneous) point of gaze and 
capitalize on the fact that visual resolution degrades with 
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eccentricity.* But much more thought is needed; it is not enough to 
discuss what might be omitted' (such as detail in the periphery) but t 
what might be included in the textured Such thought requires that one 
exkmine the visual processes in terms more analytical than simply 
"depth cues." 

A Computational Information Processing Approach ♦ / 

We now sketch^an approach to the study of vision which ha\ appli- 
cation to visual simulation. It is probably well to remark at the 
outset that this approach stands among many approaches to vision re- 
search. Later we will see how this approach, because of its different 
perspective, emphasizes certain issues of terrain depiction that have 
not been addressed previously in simulation research. 

It has long been regarded that vision embodies processes that* de- 
rive information about the visual world from the interpVetation of . 
retinal images. To be sirre*, virtually, every phrase in the previous 
statement has been contested, cast in a different light, or relatively 
emphasized by a particular theoretical viewpoint. For instance, 
Gibson (1950, 1979) characterizes human vision as direct registration 
of 'higher-order varirfles in the vis\ial stimuli, thereby downplaying 
the need for interpr«atipn and information processing (but see Ullman, 
'1979). There are some conflicting theories of visual processing, but 
largely the differences are of emphasis. For instance, some research- 
ers concentrate on the fact that our perception is often as much a 
product o^what we expect as what is objectively present in the image. 
Others concentrate on the fact that natural scenes have richly redundant 
and consistent sources of information, and argue that since human vision 
probably capitalizes on that fact, experiments using simplified stimuli 
must be carefully interpreted. 1 C~ c 

Another theoretical viewpoint is exemplified by the computational , 
approach of Marr (1976, 1978, 1981) and'Marr and Poggio (1977), which 
examines the information processing aspects of vision. The value of 
this approach to vision research stems largely from recognizing (i) that 
human vision involves complex information processing, and (ii) that 
complex systems are feasibly understood by us only when described at 
several levels of detail and abstraction. The first point is well ac- 
cepted by psychologists (in fact, some researchers find the complexity 
overwhelming an believe we will never understand Jthe processes of 
vision becaus^of their complexity). The second ptxint is well accepted 
by engineers, 'and particularly computer scientists, who see the need 
for making clean distinctions between the purposes or goals of a com- 
plex device, the methods by which it achieves those goals, and the par- 
ticular nuts-and-bolts details of how t£ey are carried out. The vo- 
cabulary used for describing what a system accomplishes is very different 



*0ne should be careful here, be$au$e while resolution degrades with ec- 
centricity; temporal processing is still strong (e.g % , tlicker and 
motion are readily detected in the periphery). 
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than that used for describing how it does that (at various levels of 
detail, culminating in the physics of transistors, or whatever com- 
prises the basic building blocks of the system). Complex information 
processing systems require this sort of multilevel description; any 
one level of description, alone, would be,, inadequate . For instance t 
the circuit diagram of a digital computer at the level of individual 
transistors and capacitors would be incomprehensible, if it;were not 
for additional descriptions of how the electronic components implement 
logical functions such as "and" and ''or," and how those functions are* 
sequenced to achieve processes, and so forth. Marr argues that bio- 
lo^ic4l r information t>roCeasing should be similarly approached/ Neuro^ 
physiology and anatomy provided us some understanding of the detailed 
architecture of the visual system, but Just as studying the physics of 
a translator reveals nothing about operating systems or Fortran com- 
pilers, neither does study of neural synapses reveal the principles of 
stereopsis or motion perception. V 

This computational approach describes visual processes at several 
levels. 5 Ideally ,*.one would like to understand some visual process 
(siicrh as stereopsis) at all levels, from the abstract and mathematical 
down to the level of neural implementation. But it ^s only for certain 
very early stages of visual processing (such as retinal function) that 
neurophysiologies! details are feasibly incorporated into a computa- 
tional theory as of yet (see Marr and Hlldreth, 1980; Richter and Oil- 
man, 1980). Of course, one fan have a significant and useful under- 
standing of a complex mechanism without* knowing its implementation > 
details. By and large, the theories emerge "top-down," w*kh the ini- 
tial insights gained, at the level Of what the visual system computes 
(regarding, for instance, stereopsis) and the theoretical basj^ for 
that computation. This sort of research is Reminiscent of Gibson and 
his followers, for one is concerned with geometric const ra^jats- and 
feasibility. But, ifl contrast, the computational approach coifcerne 
itself with the form in which visual information ^.s encoded or made 
explicit within the visual system. The 3-D representations are de- 
scribed abstractly, not at the neural level (that sott of understand- 
ing is far* from us at the present) (Marr, 1976, 1978; Marr and Nishi- 
hara, 1978; Stevens, 1980). * 4 

In formalizing- the computations that underlie vision, one makes 
rigorous the form in which the visual information is made explicitly 
the visual system, the sources of that information in the image, hdw 
that information is actually extracted and measured, and finally the 
computational constraints necessary to interpret that information. 
This sort of vision research has relevance to m simulation, therefore, 
because understanding how Information is extracted teHis us how it 



i 5 See Jlarr and Hlldreth (1980) an<! Richter *id Ullman (1980) for de- 
tailed theories of jretinal processing and edge detection. See Marr 
^>nd Poggio (1979),^layhew and Frlftby (1981), and Grimaon (1980> for 
computational theories of stereopsis. See Ullman (1979) for visual 
motion; and Stevens (1980, 1981) for texture gradients and surface 
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should be presented. ^lso, the ; constraints that visual processes In- 
corporate are essentially perceptual assumptions $bout the nature of 
the visual world. Ttmde assumptions are either always 'valid, because 
of ths physics of Solid objects, perspective, and optics, or the as- 
sumptions are usually valid because of the statistical properties #of ^ 
■ Jthe world ; see Ullman (1979) -\u>r discussion of assumptions in comAuta- _ 
ti'dnal terms and Gibson (1979) for the related notion of "ecological 
optics." These assumptions have relevance to simulation, for in laying 
out a simulated 3-D environment^ it is important to have its properties 
match our visual assumptions; otherwise, our perceptual reconstruction 
of that environment will not correspond to what was intended. A spe- 
cific example. will be given concerning the assumption^ underlying our 
interpretation of surface contours. Those results are primarily geo- 
metric; otfc^r relevant contributions to simulator research pertain to 
efficient depictions of texture. , * 

The rest of the paper reflects 'on the problems of depicting ter- 
rain for flight simulation from th& computational perspective. 

• — \ 

Shape, Orientation, and Scale' 

Let us consider in/general terms what the pilot s visual* system 
mudt accomplish/ in ord/r tfiat an aircraft may be maneuvered at low al- 
titude over actual tertrrain.^sQf course, the shape of the terrain must 
in 3-D , (so 



be*perceived in 3-D , (so that aidoBe earth-hugging course might be;fol- * 
loved. But nqt merely ^he shape\iust be known, but also its size or 
scale, A small hill anil a mountain might have the same shape^>ut- dif- 
fer significantly in sckle. We will need to become more specific about 
shape and scale, but let us continue informally for a moment. One must 
also know the pilot f ^ orientation and altitude relative to the terrain. 
The pilot, it is reported'Aenvisions himself as following a path through 
space above the ground, wh£re he keeps a mental trace of where he has 
been, where he is, and i projection in front of him of where he is going. 
In large part this mu^t^ come from his visually perceiving his orienta- 
tion relative to the terrain. We must also be specific about what this 
entails. * 

We have %ingled out shape, scale, and orientation as three classes 
of visual information necessary for flying ove* terrain. Although we 
are being informal in our use of the terms, these three notions seem 
amenable to ^precise definition eventually. But is this decomposition 
.of the visual requirements for LLP into the pe^eption of shape , .scale , 
and orientation the befit decomposition for pursues of improving the 
visual simulation? It certainly seems that ea'ctv form of information is 
basic and necessary., On the other hand, concern is, pertj^ps warranted 
that the shape, scale, and orientation are not sufficiently Inclusive; 
vfc might be omitting some different quality of 3-D information which is 
necessary for LLP. (Note that we are not considering information about 
tactical targets, navigation cues, and so forth. Rather, we are solely 
concerned with the information that must be, gathered by the visual sys- 
tem so that the pilot might «fly just ^af>ove the terrain.) We do not know 
the answer to that question; nonetheless, the shape-orientation-scale 
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approach to describing necessary information should prove useful even 
fif it is not the "whole story." _ 

The distinction between shape and scale is important . Human vision 
is often surprisingly imprecise about scale: we appreciate the shape of 
a microorganism from -a scanning electron micrograph without any concern 
for (or appreciation of) its size; likewise, an astrophotograph of a 
nebulus imparts in us no sense of ■ scale. This phenomenon is not diffi- 
cult to explain theoretically; it is likely that the internal repre- 
sentations of visual shape are inherently scale-independent (see the 
notion of scales in Haber and Hershenson (1980) and the representations 
of surface orientation in Attneave (1972) and M*rr (1978) . Furthermore, 
the visual system probably treats as distinct perceptual problems the 
determination of an object's shape and size. This distinction between 
shape and scale is often nonintuitive because in everyday scenes we 
usually have rich sources to both forms of information; there are many 
convergent perceptual processes that provide information about shape, 
and there are many means by which we directly and indirectly determine 
the actual size and distance of visible objects. The scale of our im- 
mediate surrounding area is usually so precisely known that we interact 
in it with grace and facility. But we are also faced with natural situ^ 
ations in which the scale information is relatively impoverished One 
case with which most of us are familiar concerns viewing the earth from, 
the air, where the shape of the mountains or canyons is clear to us, 
but their size is appreciated only if, say, we detect a cabin, or a car 
on a road. We then experience a sudden and sometimes shocking appreci- 
ation of the scale of the scene and, simultaneously, of the actual dis- 
tances involved. (The apparent shape remains unchanged; however, after 
we' learn the scale, it is as if independent information is added^to 
our perception.) The crucial points we should draw are two: (i) scale 
and shape are distinct forms of information about the 3-D world, and 
(ii) both forms of information are neede-d_ in order to interact precisely 

■ with the world. , i- 

The third class of 3-D information, orientation, has many facets 
and is closely intertwined with shape. First we will discuss what we 
mean by orientation and then show how shape and orientation relate, 
Gibaon (1950) discusses two qualities of orientation; one local, the 
other global. The local orientation of surfaces is defined -relative to 
the viewer; one's orientation in space is defined globally relative to 
the surrounding visual environment. Let us consider each in turn. It 
is natural to visualize the orientation of a patch of surface relative 
to oneself. The slant angle between the line of sight and the normal 
to the surface is one way of quantifying this. When viewing a large 
planar surface such as the ground' seen from the air, there is a wide 
range 6f relative surface orientation (zero slant directly below the 
aircraft, slant approaching 90 degrees toward , the horizon). But we 
. think of the planar surface as having one orientation, not merely a 
local orientation that depends on which patch of the surface we view 
If the plane rolls, we see the orientation of the ground change relative 
to us. We clearly have the ability to judge orientation more globally, 
e.g., relative to the horizon. 
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We have discussed two sorts of orientation information: informa- 
tion about the attitude of individual patches of surfaces relative to 
the viewer, and the overall orientation of a large surface such as the 
ground seen fr om th e air. There is ano ther related sort of information 
which also seems primitive and important: the direction of movement. 
Whether running through a forest or flying at low altitude, we are , 
visually aware of which way we are traveling relative to the 
environment. 0 V 

If we regard -shape, orientation, and scale as three types of 3-D 
information necessary, for flying relative to the terrain, we may ap- 
proach the problems of improving the terrain simulation in these terms. 
The approach is as follows. . 

The Basic Approach *. 

The visual deficiencies of-the.LLF simulation would be cast in 
terms of shape, orientation, and scale. In other words, one would as- 
certain where the performance problems Vie in simulate^ LLF flight, to 
what, extent the cause is inadequate perception of the shape of the ter- 
rain (Is that ridge sharp or is it smoothly rounded? What does the 
terrain do over to the right?), inadequate perception of Orientation 
(Is the aircraft yawing? Is the ground ahead rising or horizontal?), y 
and inadequate perception of scale (How high am I above the ground? 
How far is the ridge?). Insight into the underlying problems cast in 
these terms might be gained by many means. Pilots might even be ver- 
bally examined in these terms, because we all have a strong (and simi- 0 
lar) understanding of what is meant by shape , orientation, and scale. 
It was merrtioned earlier that pilots commonly remark that* judging al- 
titude is difficult. That, for instance, is a matter of sc^le. 

In addition to direct interview, there should be more quantitative 
psychophysical means foil' determining what 3-D information is deficient, 
but again, cast only in- terms of shape, orientation, and scale, without 
attempting to uncover what "depth, cues" are missing from Ehe CIG display. ^ - 

Note that we seem to attack what has been a thorny problem with 
only v6ry informal and <j: asua l terminology. But the rigor comes later, 
when we apply knowledge about visual perception to suggest, improvements f'^ 
to scale, or whatever. We will see an example of this analysis when 
we discuss how shape is perceived from undulating surface contours. 
This would have importance in the CIG depiction of rolling terrain. 
First it will be useful to examine a wide range of visual processes 
so that we begin to sort out«which processes concern shape, whic^i con- 
cern orientation, and so forth. In a sense, we are abandoning the 
rather simple no'tion of "depth cue" for three types of surface infor- 
mation, not just depth. This has proven •useful'yin vision research 
proper. To illustrate, stsreogsis is not: merely a "cue" to depth but v^j 
also of shape, at least; shading also provides shape information but 
tells us nothing about scale (the shading of the moon is similar to 
that of an orange). We cannot expect a simple correspondence between 
process and either shape* or orientation, or scale— vision is far too 
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intertwined to be cast in such simple terms. But on the other hand,, if 
we know that we need additional information about shape, we have many 
visual means for providing that information. . It goes without saying 
that our understanding of vision is not sufficiently compete as t.o ; ; 
provide ready-made recipes for improving the simulation* , ; . 

As a final introductory remark, note that although we might treat 
scale, orientation, and shape as substantially independent topics , we 
will probably find that improvements to the CIG display intended to 
enhance, say, the perception of shape will also improve the apparent 
orientation, and so forth. These side effects would be welcomed, of 
course. But the general strategy is to understand how one gains a 
perception of shape, orientation, and scale in order to increase the 
effectiveness of the simulator in inducing those perceptions. 

Shape: What Are the Sources? 

There are several potential sources of information about the shape 
of a surface,- including binocular disparity from stereopsis, visual mo- 
tion, shading, texture gradients, and surface contours. Let us consider 

each in turn. . 

First, although stereopsis provides an acute sense for 3-D shape 
in our usual environment, it is not useful for large-scale shape per- 
ception, e.g., of terrain, a few miles away. The reason is that only 
the very near terrain causes detectable stereo disparities, and since 
that portion of the scene is also moving across the field of view at 
high angular velocity, one must track some nearby surface feature; 
otherwise it is blurred . On the other hand , stereopsis cannot be ruled 
out as potentially determining scale , especially in the foreground (see 
later) . 

Mot ion: Parallax. Optic Flow, and Shear Visual motion P rov ^es 
one of the dominant sources of information about shape. Several differ- 
ent aspects of motion have been distinguished, such as shear (dis- 
continuities in projected angular velocity which arise, for example, 
along the edge of, a physical object seen against a relatively distant 
background), "optic flow" (the wide^field visual effect from movement 
through the environment), and "motion parallax" (the changing projec- 
tion of an object as it moves relative to us, from which we^Lnfer its 

3-D shape) . ' . . 

Before discussing these aspects of motion, we reiterate that 
visual motion across the CIG display comes automatically when the 3-D 
model is transformed by perspective projection while ' continuously 
(i.e., at a sufficient rate) updating the viewer's position in that 
model according the speed and direction of motion relative to the 
model. Thus motion per se is given; it is what is in motion across 
the CIG display that is relevant to our discussion. For instance, are 
moving dots sufficient? (Luminous dots are particularly 'cheap to 
display on many CIG Systems, whereas surface patches are in relatively 
short supply.) 
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In fact;, moving dots are effective: if collection of 3-D 
p'oints in some rigid arrangement {such as the runway marker lights 
and the lights of nearby buildings and streets) are projected in mo- 
tion on the display as if we were moving relative to them (as in 
landing), the visual interpretation in 3-D is retoarkably precise.** . 
Of course, the designers of visual simulator night displays have long 
been familiar with this. Note the 3-D shape is accurately perceived, 
but not the scale, unless there are 'familiar cues such as the runway 
width (see later). 

So we know that points are sufficient stimuli, but what about 
lines and edges? What do they contribute over mere points of l^ght?. 
This perceptual issue has not been fully unraveled , but some relevant 
observations may be made. Our visual processes, seem to require dis- 
tinct and traceable image points in, order to derive a 3-D shape, but 
a curve or line only offers* the endpbints or points of discontinuity 
in tangent ; arbitrary intermediate points along the curve cannot be 
tracked (see Ullman, 1979) . Incidentally, this suggests that only 
the corner points. Of the checkerboard pattern (popularly used to de- 
pict terrain) contribute useful input to the motion interpretation 
process J 

When traveling across a textured surface there is ; an apparent 
streaming of detail across the visual field, commonly termed optic 
floy . This is regarded as a wide-field phenomenon, while the previ- 
ously discussed motion parallax is detailed and foveal. What 3-D in- 
formation is derived from optic flow in the periphery? Some hypptheses 
have been forwarded that the optic flow specifies distance up to a 
scalar (Gibson, 1950; Nakayama and Lpomis, 1974) and even local surface 
orientation everywhere across the image (Ko en d e r ink and van Doom, , 
1976). Those two proposals suggest that optic flow provides shape (but 
not- scale) infonjiation, but they are largely theoretical; it remains to 
be seen What sort of 3-D information is derived, in fact, by the human 
visual system. It is possible that negligible shape information is de- 
rived from the periphery, even under the best conditions. Optic flow, 
particularly in the case of LLF, may provide only information about 
orientation and the direction of travel (this is discussed furthe* * 
later) • . . 

Another aspect of visual motion is motion shear . Whenever an 
opaque surf ace^ feature protrudes above the mean surface level, it oc- 
cludes from view that which i$ behind it. Occlusion thereby provides 



See Ullman (1979) for a theorem showing how, f this interpretation is 
feasible solely by analysis of the images without requiring higher- 
level knowledge. « 

^But that does not mean one can replace the checkerboard with dots 
(placed where each corner was) and have an equally compelling 3-D 
impression. The checkerboard also provides a useful texture gradient, 
at least. This demonstrates a difficult aspect of this work: visual 
processes and their visual inputs are largely intertwined . 
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information about relative distance-it allows us to 

visual world feeo surfaces segregated by distance This is -^J"" • 
larlv striking in LLF when the relative motion of nearby ridges against . 
^ background 8 terrain causes them to be seen in relief . Since sheat 
contributes to the apparent terrain shape , it deserves s udy I^par- 
ticular, the questions of what is shearing and whether it is a simple 
dot pattern or edges or lines is of importance to CIG design. The 
percepti^of 3-D shape from motion was only briefly sketched here 
lt"s ho^ed that it shows that while studying the perceptual processes 
one can also- derive specific implications for CIG display. 

Shading. Texture r.^gnts. and Contours ,We have ^fj^j^ 
dynamic aspects of the visual display. The other side of the coin, is 
the static properties which carry information about shape. These in 
cJude shading, texture gradients, and contours that lie Across the 

8Urfa The apparent 3-D shape of a surface is usually enhanced when ^ 
ohaHp ; The interpretation of shading information is analytically a 
very difficult Problem requiring knowledge of the illuminant directions 
^he y reflectance P functions of the various surfaces and means for dealing 
with complex shadowing and mutual illumination conditions (Horn . 1975 . 
But the human visual system probably ddes not attempt such a solution, 
instead iHxtracts weaker inferences of 3-D shape that would be true 

Sid to Tund«sLnding of Just how si^le .„ approxt-tlon to shading 
can be effective in the simulator display. ' . 

Another source of 3-0 information, which we will discuss only 
brief^ is the texturegradient. A homogeneous distribution of physi- 
caJ texture acr oss a sur f ace resu lts in a texture gradient in the image. 
Because of perspective projection, the image texture in every locality 
is both foreshowed and scaled. We can recover shape ^J^w 
(specifically distance up to a scalar) from the texture gradient but 
L discussed earlier, probably not from texture density. Further re 
search's needed to determinehoW texture gradients are measured in 
order that we may gain insight into how best to depict texture for CIG 

di8Pl Contours that lie across a Surface, as depicted in figure B r la, are 

those assumptions are not violated when modeling terrain by means of 

'^Observe in figure' B- la that the lines appear £££ 
lating surface/The basic assumption that we make is that the curvature 



of the lines re fleet a surface curvature. 9 It is as if the lines were 
straight lines on a carpet that became curved because the carpet was 
rippled. More formally , these contpurg are geodesies with the addi- 
tional property of being lines of greatest curvature (see Hilbert and 
C&hn^Vossen, 1952, for a lucid discussion of these concepts). 




FIGURE B-l The curved surface in (a) resembles a rippled carpet. The 
straight lines lie parallel to the ridge, the curved lines 
are perpendicular. The curved lines are lines of greatest 
curvature and planar; the straight lines are lines of least 
curvature. Note that the curved lines alone tell us a 
great deal about the shape of the surface (b) , but the 
straight lines alone carry virtually no information. 



A line of greatest curvature may be thought of as a path across a 
surface which experiences the greatest undulation. Note that in fig- 
ure B- lb the straight lines that follow the ridges are perpendicular to 



Note that we do not easily see figure B-l for what it is, a collection 
of lines on a flat sheet of paper. Instead, we interpret the undulation 
as having been caused by the underlying surface. 



■^iL^Z* uresis n« i»orASf gives us constraint o„ the 

™cr^^ 
will riot "be elaborated upon here) . 

1 vTVip nhtfsical curves are. lines of curvature. ; ... 

2 Stersect^g curves «.t at a right angle on the surf see ea a ; 

V.V J.. Xre'tnefcface'contonrs are p.r.11.1. the eurfece;ie singly 
curved (like a rippled carpet). 
• 4 Seoause of CD and (3) the corves ate geodesies and planar. 
The ah^rovlde^tne' liaelv hasia t pr our ^^l 

■~JSLS? o^oeaf ™£T.S£S » - SSS' h UroUy 

rur^W^r esaion o.-shape }f consist SLSSU%TS» 
b " ^ISX terrain, "PeciaXly 'undulating terrain it ia — • 

ln„lng restrictions:^^ ^ . . ^ , ( cury.tur. jnd geodesic. 

9 Anv intersecting curves shouid meet at a right angle. 

V U the aurf ace is singly curved ,*like a smooth ridge, it is 

3 ' 111 iVZllry to depfct a few lines of .•"•^ c-^r. 
(as in figure B-la) . TJie Unes parallel to the ridge may 
omitted in general. ± 

4 If the surface is doubly curved, like a round hill, it is im 
portant to place lines of curvature at close spacing.^ It is 
best to include both sets of lines of curvature, so the sur- 
face appear^ to have a net draped over it (figure B-la). E.<* 
intersection will then be a right angle, and the surface shape 

These are^few'rulef iTsn'ofid be followed. Not. that they may be 

following a line of greatest curvature (the fall line as 
««. Q «/i rhflf the curve is not what we assumed. 

'"^Depicting surface shape by undulating linea as Just discussed 
aUould be compered with the method of using checkerboarda. Firat 



fti 



all, checkerboards have been proposed because they are relatively eco- 
nomical ijj terms Qf edges, and squares seen in perspective are easily 
interpreted in, 3-D. A checkerboard pattern, in fact, is probably opti- 
mal for depicting a planar surface, gi ven the trnrrent i±mi t a t i onsr ~ifi 
displayable edgefe. But the Straightforward extension of checkerboards 
to indulating surfaces may not be the best approach. First let us con- 
sider .singly curved surfaces — ripples in a carpet. Checkerboard pat- 
terns are useful for ridges so long as the rows are arranged parallel 
with the- ridge. In that case we have the geometric arrangement of 
figure B-lb where the edges of the squares that are parallel with the 
ridge (in other words,' those that lie at constant elevation) are lines 
of least curvature, and the other edges climb over the ridge. But as 
stated earlier, the lines of least curvature may probably be omitted 
because they are redundant * In terms of edges , then, a "series of 
stripes, rather than checkers, would be more economical and probably 
would be equally effective. For a doubly curved surface such as a round 
hill, the problem becomes somewhat more difficult, but it is solvable 
if sufficiently f'ine stripes or checkers are used. That is because 
a doubly curved surface, if sliced fine enough, can be treated as singly 
cttfved in general.] Incidentally, the line defined by a column or row of 
checker squares has alternating contrast along its length. This tends 
to' break up the lines, for the visual system has difficulty in aggre- 
gating into a litie elements with opposite contrast sense. The checker- 
board pattern is unlikely treated as a collection of long surface con- 
tours, therefore. Instead, each white or black square is seen disjointly. 
For that reason, long stripes or lines may be more useful in depicting 
undulating terrain. 

Sources of Information About Orientation 

'The previous discussion covered the major sources- of visual infor- 
mation about surface shape. Next we will discuss orientation. This 
discussion is more difficult, because 3-D shape is most naturally de- 
scribed relative to the viewer and therefore in representing shape one 
simultaneously captures several aspects of orientation. For instance, 
recall that in the night landing simulation, motion parallax involving 
mere luminous dots on a dark background can give a compelling impres- 
sion of movement toward the solid earth during landing. Clearly we per- 
ceive the orientation of the terrain as well as its shape. While orien- 
tation and shape are intertwined, it is useful to approach visual 
processing from one or the other perspective, depending on the need. 

The direction of movement and one's spatial orientation are two 
forms of orientation information that are relatively distinct from 
shape. Both are difficult to visually determine in LLF since uneven 
terrain often prevents one from using a distant horizon as an altitude 
reference. But optic flow seems useful (Gibson, 1950), particularly' 
in LLF, where the terrain appears smeared anfl blurred (see Harker and 
Jones, 1980). However; the highly simplified CIG displays might not 
provide sufficiently dense surface texture moving with sufficient 
smoothness of motion to be effective. Again, we see that further 
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research is needed to see how' optic f 16V is visually processed. This 
-research problem is of particular importance to LLF. A final point 
regarding optic flow should be made. Simply because the ground tex- 
ture appears blurred and streaky in LLF does not mean that one can 
display a simple pattern of streaks .in the- periphery displays. When 
one tracks a detail on the ground, it becomes roughly stationary on 
the retina and the smearing of detail vanishes. We do not know What 
role tracking eye movements would play in LLF (certainly they are im- 
portant when scrutinizing/ground targets, but what importance they have 
in the basic problem of flying LLF is not yet known) . 

A local "cue" to orientation which has been incorporated in CIG 
displays is the Simple geometric, arrangement of two lines or edges that 
intersect at right angles on the surface (figure B-2a) . It ^straight- 
forward to show that the image of a right angle (which because of fore- 
shortening in the perspective projection appears .as an obtuse angle) 
carries some information about the surface orientation, but It alone 
does not specify a unique 3-D orientation,; it could theoretically cor- 
respond to a large variety of differently oriented planes in 3-D. Ad- 
ditional constraint comes from assuming that the two lines are eoual 
length in 3-D. 9 Note that these two constraints— perpendicular inter- 
sections and equal length segments-are neatly combined in the square 
checkerboard patterns which are often used in terrain simulation. 
Perpendicular intersections in general are useful indicators of sur- 
face orientation, provided the lines or edges are of equal length on 
the surface. _ 




a ,b 





FIGURE B-2 The intersecting lines in (a) appear to lie on planar sur- 
faces in 3-D. The interpretation involves assuming the in- 
tersection is perpendicular and the two lines are of equal 
length on the surface. The intersections in (b) may be 
thought of as a signpost at a street intersection seen from 
different viewpoints. To interpret this configuration, it 
is necessary only to assume the intersecting lines are mu- 
tually perpendicular in 3-D. 



9 There is still an ambiguity of the Necker cube variety. But when view- 
ing this configuration within the larger context of the ground plane, 
that ambiguity is resolved. 
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A point that has seldom been emphasized Is that a trihedral inter- 
section, three mutually perpendicular lines (figure B-2b) , actually car 
ries enough information in its image'to fix its surface orientation 
precisely. i^ The arrangement resembles a signpost at a street inter- 
section — note that the line segments no longer need to be equal length 
in 3-D. This configuration would be a very effective means for indi- 
cating orientation, kven scattered across an otherwise featureless 
terrain. 



Sources of Information About Scale 

Now let us suppose that the terrain simulation is adequate in terms 
of shape and orientation. The problem "that would remain is to ensure 
that the pilot can judge the distance to the surface and his altitude 
above it with sufficient accuracy. What are the sources of information 
about scale that he might utilize? Scale may be inferred by several 
perceptual methods: directly f rom* stereopsis or indirectly from, the 
known size of recognizable objects, from known velocity, and several 
other methods that we will discuss. Scale is certainly necessary for 
various flying tasks'; the difficulty that pilots often have in landing 
amphibious planes on water and maintaining low-level flight over water 
and sand dunes suggests that there are real visual scenes that are de- 
ficient in information about scale. 

f Stereopsis Stereopsis may, in theory, have a minor contribution 
at low altitude, but to see this we need to be somewhat quantitative 
for a moment. Of course, stereopsis is most useful in the very near 
environment (distances of less than 30 feet), but we are sensitive to 
stereo disparity but to roughly 1,800 feet (computed from a stereo- 
acuity of 24 arc seconds (Graham et al. , 1949)). Hence wfe cannot dismiss 
the possibility that stereopsis provides a pilot altitude information 
when flying LLF (some pilots repdrtedly look out of the side canopy to 
judge their altitude (Barker and Jones, 1980)). At roughly 100 feet in 
altitude, one can see downward sufficiently well to be viewing surface , 
detail within the utteful range of stereopsis (Kennedy and McKechnie, 
1970), but because of the blurring and streaking, which we have dis- 
cussed, unless the pilot tracks some feature on the ground, stereopsis 
would probably fail. But a significant impression of depth might be 
derived, provided one tracks a £utface feature for sufficient time to 
achieve stereopsis before it is out of view. Consider an aircraft 
traveling 500 knots at 100 feet and a surface feature that passes 
within 500 feet of the aircraft at nearest approach. One may easily 
compute that approximately two seconds elapse from the time the point 
enters stereo range (1,800 feet) to when it passes directly by the side 
of the aircraft. Hence, if a pilot in low-level flight has the time 
to visually track a surface feature during tKe time it is within stereo 
range, stereopsis could conceivably provide useful scale range. It 



Again, up to a Necker reversal which can be disregarded here. 
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must be stressed, however, that head movements would be required to . 

track the fnirffl" 0 feature, the visual trackingHask wo u l d n ot b e e as y 

in a hostile environment, and stereopsis under those visual conditions 
has not received sufficient psychophysical study. 

There is evidence that casts -doubt on this hypothesis, however, 
The evidence is simple: recall that certain types of terrain are par- ^ 
ticularly difficult to fly over at low level— specif ically , sand dunes 
and open water. There is sufficient viewing time to achieve binocular 
fusion, sufficient visible detail on which to establish stereo fusion, 
and a large region of the surface is within stereo range at low alti- 
tude. Nonetheless, LLF is difficult over that terrain. Apparently 
the sense of scale (if any) that one derives from stereopsis is insuf- 
ficient for making the critical altitude judgments. Stated another 
way, stereopsis may play some role in LLF, but it is probably insig- 
nificant. It must be said, however, that the above is not conclusive-- 
the issue -deserves careful experimental consideration. 

Known Size One method to infer distance is from the measured 
retinal size of an object of known size. It is well-known that the 
distance to an object varies inversely with the angle it subtends. 
Theoretically,, if one knew that a tree- were 20 feet high, one could 
also know its absolute distance on the basis of its retinal size,. 
Certainly the visual system capitalizes on this relation, 'and we even 
consciously search a novel scene for some cue to its scale (conscious 
attention is not required, as Helmholtz (1925) recognized in his term- 
ing it "unconscious inference"). There" are phenomena which strongly 
suggest that we do infer distance fron known size (e.g. , Enright, 1970).- 

However,. empirical studies of "size constancy" and absolute dis- 
tance perception (Epstein* and Landauer, 1969; Gogel, 1971; Hochberg, m» 
1971; Rock and McDermott, 1964) have shown that the psychophysical re-^ 
lation between distance and retinal size is not as simple as might be 
expected. This is one place where simulator designers have experienced 
difficulty in interpreting the psychological literature. Many of the 
experiments were performed in artificially restricted viewing situations 
(e.g., darkened rooms with few reference objects) and those that were 
performed "in the field" would involve verbal judgments of distance, 
e.g., as a target being so many feet away (Gibson and Bergman, ,1954) 
or comparison judgments between two distances (Foley, 1972). Few ex- 
periments reveal just how precisely we perceive, the scale of the- visual 
world from objects of known size. 

It must be stressed that we are ultimately concerned with the 
visual judgments of absolute distances, and therefore with providing 
sufficient information so that the pilot can fly 100 feet above the 
terrain, for instance. The known-size method probably plays a role in 
this ability. This is another place where tightly directed investiga- 
tion is critically needed. , 

Known Velocity A potentially useful method for computing scale is 
quite similar to the known-size method just discussed. If one knows 
the velocity of travel past an object, one can infer the distance to 
that object from the induced angular velocity (e.g., retinal velocity 




given a fixed gaze, or velocity relative to some point on the cockpit 
canopy). The following relation between absolute distance d f absolute 
velocity V, and angular velocity u was pointed out by Gibson and others 
(Gibson, 1950; Nakayama and Loomis, 1974): ? 

Vsing 

d - = (1) 

where 8 is the angle from the direction of travel to the' given point 
whose distance is to be measured. The relation presumes that the given 
point is stationary and the viewer is in pure translation. It is 
powerful in that it allows one to compute a depth map for the entire 
visual field (everywhere there is detail moving across the field of 9 
view) in terms of co and 6 (measurable, in theory) and the single un- 
known. V, That depth map would specify absolute distance if the abso- 
lute velocity AT were known. 

Conscious deduction is not necessary or even likely in this pro- 
cess. Instead, a pilot experienced in flight at low altitude and high 
velocity might come to expect a particular angular velocity in the 
periphery, in a manner analogous to driving a car and expecting the 
road to slip by at an appropriate rate for any given driving speed.* 1 
Then if the groundspeed is held constant but the altitude is lower than 
normal, the angulaenuelOfiity would be higher than normal, and vice versa. 
This relation could, th6orfestically, account for the precision with 
which altitude may be held rfr LLF by experienced pilots. It is not 
inconceivable that part „of tha skill that a pilot acquires through 
flight training is the unconscious calibration of retinal velocity ac- 
cording to ground speed and altitude, and furthermore the development 
of an effective feedback loop that attempt* ^ to control altitude accord- 
ing to actual retinal velocity at any instant. 

It should be relatively straightforward to establish whether this 
is the case. The direct relationship between altitude and groundspeed 
predicts that as speed increases there should be a tendency to increase 
the cruise altitude, other factors being constant. The following evi- 
dence (J. Richter, personal communication) suggests that this occurs. 
Interestingly, it is most apparent 'when flying. over open water (the 
situation we noted before as difficult)^ Perhaps the fact that water 
provides little evidence of scale, compared to a richly textured rural 
terrain containing familiar surface features, allows one to observe 
the weaker contribution of the known speed method. Over the ground, 
one's altitude tends to remain fairly constant as speed increases, but 
over water the altitude definitely tends to climb as the aircraft is 
accelerated. It is consistent with there *being some unconscious 



It is noteworthy that when changing from a passenger car to a low 
sports car the apparent speed is greater merely because of the larger 
angular velocities associated with being nearer tne ground. 



process attempting to maintain an expected angular velocity, where an 
increase in aftgular velocity is attributed to a loss of altitude ra- 
ther than an Increase in ground speed. , These informal reports suggest 
that the knovfi velocity method for computing scale may have some 
validity in abtual flight, and therefore should be given careful ex- 
perimental investigation. The experiments should probably be performed 
using unfam/liar surface textures to examine the relationship between 
speed arid^filtitude. The problem must.be addressed in conjunction with 
an inve^igatiop of texture, as undoubtedly the effect will vary in 
magnitude with texture variations. 

Other Methods In addition* to the two methods just discussed, 
there are other geometrical methods for gaining scale information. 
Several have been discussed by Harker and, Jonee (1980). Typically they 
consist of some geometric quantity— e. g. t the ratio of angle subtended 
by some vertical surface feature compared to*its angular distance be- 
low the horizon— from which one may judge whether the altitude is main- 
tained constant, or whether the aircraft will clear a vertical obsta- 
cle. 12 While strictly speaking these geometric quantities do not * 
specify scale in the sense we mean, they are, nonetheless, potentially 
useful in flying. We probably should distinguish between qualitative 
problems that can be solved geometrically (suchas obstacle clearance 
and maintaining constant altitude) and quantitative problems (such as 
clearing obstacles by so many feet, maintaining a particular altitudeT^— 
and so forth). But it is not- certain which sort of problem, the quali- 
tative or the quantitative, deserves the greater attention at the moment. 

Sketch of an Application of the Computational Approach Q| 

« 

To illustrate a specific example of using the computational ap- 
proach to improving the CIG simulation of terrain, consider the depic- 
tion of undulating terrain for LLF. We focus on the problem of improv- 
ing the visual information about the shape of the terrain, which of 
course is but one aspect of the overall effort of generating an effec- 
tive LLF visual simulation. * 

Reflecting on the various sources of visual information about 
shape, we recall that these include visual motion (of surface features 
due to one's movement over the terrain), shading, texture gradients, 
and surface contours. Suppose we focus more closely on the contribu- 
tion that surface contours make to the apparent shape of the terrain. 
That is, Jet us consider how one might make terrain features such as 
ridges or valleys visually apparent by essentially paintirig contours 
(perhaps depicting property boundaries or roads) across the CIG terrain 
model. — 



Some are well known and supposed to be consciously attended to. For 
instance, if the top of a tower is climbing in the visual field it is 
above the flight path and will not be cleared. 



( 
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The results of theoretic analysis of surface contour perc^ption^ 
were sketched earlier. Basically, our 3-D interpretation of dbntours 
embodies particular assumptions about the geometry of these cotttovirjs 
and their relation to the surface across which they lie* The ^sump- 
tions include (see earlier and Stevens, 1981): 

1. The physical curves are lines of curvature. 
* 2. Intersecting curves meet at a right angle. 

3. Parallel contours imply the surface is singly curved. 

4. The physical curves are geodesic and planar. 

If thoge geometric assumptions are indeed those governing human percep- 
tion of contours, that suggests a means^for placing contours across CIG 
terrain models so that they are visually effective (recall the fo«r "de- 
sign rules" for surface contours described earLier). > 

Thh next step, therefore, is careful psychophysical study that, 
verifiers that surface contour perception involves the geometric assump- 
tions that were established on purely theoretic grounds. Specifically, 
consider item 4 above. If our perceptual- apparatus incorporates the 
assumption that a viewed curve is planar and geodesic (in the absence 
of information to the contrary as provided by, say, stereopsis and mo- 
tion), that assumption has predictable and testable consequences. What 
would be required next, therefore, is. empirical investigation. Once we 
have established the fole of these (or other )geometric assumptions, a 
quantitative study is required of the relative importance of st*ch factors 
as contour spacing, line width, and so forth. 

■ ■ * N ' ' 

X ...» 

Concluding Remarks 

Three types of 3-D information that must be gathered by the visual 
system in order that a pilot might fly cl&se to the terrain are its 
acale, shape, and orientation. The visual deficiencies of the LLF simu- 
lation can be discussed in such terms. Indications are that sc^le in- 
formation requires the greatest improvement. 

More rigorous analysis comes,jnext. 4 We gain insight into improving 
the ^display by learning how theftuman visual system determines scale, 
for example. There^ are mamKsouTces of * scale information, but much 
more effort is required before we can make concrete suggestions' regard- 
ing CIG improvements. 

We did see an example of where the computational analysis o| sur- 
face contours, a* source of shape information, leads to rather specific 
suggestions regarding the visual display. For our visual systems 'to 
make sense of image curved in terms of actual contcfurs lying across 
physical, surfaces, a number of geometric assumptions have to be made. 
Analyzing what these might be, in theory, coupled with psychophysical 
verification that those particular assumptions are involved in our in- 
terpretation process and not some other set of assumptions, leads to 
some design rules, aj it were. The design rules are intended for the 
simulator engineer who^depicts terrain using curves (one curve might be 
meqnt to depict, a fence over a hill, another might depict the edge of 
-a field). It is of course important that the pilot perceive the simu- 
lated world' in the way that it was intended by the designer. To do so, 
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the contours placed in the terrain mpdei should be restricted in their 
geometry so that they match the geometric assumptions that govern their 
visual interpretation. 

The surface contour example is a particularly clean demonstration 
of how understanding some aspect of vision has application to simula- 
tion. As. should be evident from the previous discussions, much needs 
to be learned about vision. „ 
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APPENDIX C 



VISUAL SENSORY ASPECTS OF SIMULATION 
One Example of an Experimental Approach 
D. M. Regan 



Rationale 

Several quite different strategies have been Suggested for coming 
up with visual factors that might correlate with flying performance in 
a simulator or aircraft. These strategies range from multivariable 
factorial studies to the "natural computation" approach of the MIT 
group. This appendix outlines one candidate strategy for identifying 
visual sensory factors that are important in flight simulation.. This 
approach runs along the following lines: 2 

1. Carry out basic' research on the. sensory aspects of vision, J 
testing some specific hypotheses about how the eye processes 
visual sensory information . 

2. From the results of this basic research, predict which visual 
functions will be important in simulation. 

3. Test these visual functions in pilots, and correlate the 

test results with simulator (and aircraft) flying .performance. 

In order to focus our efforts, we have restricted ourselves to one 
of several possible hypotheses as a guide. The next section of this 
paper lists some of the resulting basic research findings and conclu- 
sions that might be relevant for simulation. ' The last section outlines 
two follow-up studies of simulator flying performance. 

We should note that the theoretical approach used in these simu- 
lator studies is no more than a working hypothesis or guide. 



1 lt is often noted that, in general, an increased basic understanding of 
visual function would help to improve simulator design. This appendix 
can be read as an account of one (among many) practical ongoing efforts 
along these lines', in which the same experimental thrust has been main- 
tained over a ten-year period. The simulator studies outlined here 
could by no means have been carried through without generous help and 
advice from staff of the U.S. Air Force Human Resources Laboratory, 
Williams AFB and all flying personnel involved for Williams AFB, Nellis 
AFB, Luke AFB, and MacDill AFB. 

2 We appreciate that other, quite different, strategies have been used in 
the bulk of visual research on simulation. 
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Oux theoretical approach has been firmly In the camp of those who 
use simple laboratory stimuli, often at threshold levels, with the aim 
of uncovering elementary functional subunits in the visual pathway 
(Braddick et al. , 1978; Regan, 1982). If we wish to usefully predict 
vision in a complex visual environment, we should search for subunits 
that analyze retinal image information into more or less orthogonal 
(nonpverlapping) elements. However, it is not necessary that a^given 
subunit be linear • 

Examples of Basic Research Findings and Concisions with 
Possible Implications for Simulator Design 

\ t 

The optical flow pattern generated by an observer's movement con- 
tains information about the direction of movement (Gibson, 1950, 1958; 
Richards, 1975). It has been sometimes suggested that an observer could 
determine his projected impact point from the center of expansion of 
his retinal flow pattern; however, there is reason to think that in 
practice this cannot be done (Llewellyn, 19F71; Regan and Beverley, 
1981). If the observer fixates his gaze off to one side away from his 
direction of movement, the retinal flow pattern will be distorted 
(Regan and Beverley, 1981; Richards, 1975). On the other hand, .subjects 
can accurately judge where, on the retinal image, the local magnifica- 
tion is changing most rapidly, and in some situations this direction 
coincides with the direction of self-motion (Regan and Beverley, 1981). 
There is evidence that visually judging the direction Q.f self-motion on 
the basis of visual flow patterns may involve selective sensitivity to 
changing size (Regan and Beverley, 1979a). 

In order to explain certain experimental data, we theoretically 
proposed that the human eye responds to an objects rate of change of 
size more or less independently of the object's motion in the frontal 
plane (Regan and Beverley, 1978, 1980). Note that an object's rate of 
change of size is closely related to its line-of-eight component* of 
velocity. In particular, the response of the eye's chahging-size sub- 
system may directly indicate time to collision, more or less indepen- 
dently of the observer's trajectory and of the object's trajectory. 3 

The human eye processes stereo position in depth separately from 
stereo motion in depth; in addition to the classical stereoscopic depth 
mechanism sensitive to disparity, there is a second stereoscopic mecha- 
nism sensitive to the relative speed seen by the left and right eyes. 
Physiologically, there are different types of brain cells predominantly 
sensitive to disparity and stereo motion in depth (Regan et al., 1979; 
Richards and Regan, 1973). , 

A sensation of motion in depth can be produced either by stereo- | 
scopic (binocular) stimulation or by changing size. Relative visual * 
sensitivity to the binocular and to the monocular cues does not much 
depend on viewing distance (contrast with classical stereo vision). 



3 That (Time to Collision) - (Object 's Fractional Rate of Change of 
Size)"" 1 is easily shown for a nonrotating object. 



Relative binocular to monocular sensitivity does, however, depend on 
the object's speed and linear size. This suggests that there is no 
general answer to the question* "What effect has closing one eye upon 
a pilot's ability to judge motion in depth?" (Regart and Beverley, 
1979b). 

Findings suggest that for motion-in-depth perception, static 
texture is worse than no texture (Beverley and R£gan, submitted). 



Correlations Between Visual Test Results and 
Simulator Flying Performance 

This section describes two studies in which the results of visual 
tests on pilots were compared with flying performance on a flight simu- 
lator (A-10 aircraft) and with aircraft flying grades (T-38 jet train- 
er) at Williams AFB. 

In the first study (Kruk et al. , 1981), subject groups; comprised 
twelve pilot instructors, twelve nonpilot weapons officers, twelve 
graduating student pilots, twelve first-year student pilots, and twelve 
nonmtlitary nonf lying Subjects. A total of thirteen visual test mea- 
sures were collected from •each subject, including manual tracking of a 
target that moved in depth in the presence of vibratory "jitter" motion 
(Beverley and Regan, 1980). The simulator flying task was restricted 
visibility landing. In brief, the better the pilot's performance in 
tracking a target that moved in the frontal plane, the earlier was the 
first visual flight correction during landing. In addition, the number 
of crashes was predicted by tracking accuracy for a target that moved 
in depth. Since the only difference between the two tracking tests 
was visual, motor factors being common, and since the two tests correMfc 
lated with different aspects of landing performance, we concluded tha^P 
the two tracking-tests detected intersubject differences in visual sen- 
sory factors that are important in landing (though intersubject dif- 
ferences in eye-hand coordination may well have been involved also). 
Quite different correlations were obtained between landing performance 
and psychophysical thresholds for motion and contrast. Since these 
correlatipns were weak, we concluded that these threshold tests failed 
to detect the performance-related intersubject differences in vision 
detected by the tracking tests. 

Different individual subjects were used in a second studg,. Simu- 
lator flying performance was compared with the results of sensory visual 
tests for twelve experienced fighter pilots (group 1), twelve pilot 
instructors (group 2) , and twelve graduating student pilots (group 3). 
The following *is extracted from a report submitted for publication 
(Kruk et al., submitted for publication). 

Flight Simulator Tasks 

The A-1Q cockpit of the Advanced Simulator for Pilot Train- 
ing (ASPT) at Williams AFB was used throughout. The simulator 
was initialized 146 m behind an A-10 lead aircraft and the pilot 
instructed to close and assume fingertip right position until 
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the simulator re-initialiaed after 2 min flight. This pro- 
cedure was repeated for close trail and fingertip left posi- 
tions, then the whole sequence was repeated with the horizon 
removed. Time spent in correct formation was measured. 
Groups 1 and 2 then flew a low-level exercise. Pilots were 
instructed to fly through a mountain pass and on for about 
5000 m while maintaining an altitude of about 36 m, then pop 
up to about 360 m and execute a dive bomb attack on a speci- 
fied target (one of five buildings near a road). Then the 
pilot flew about 3000 m at about 36 m altitude to exit the 
area*. There were two SAM sites and two active antiaircraft 
artillery sites. Auditory .tones indicated to pilots when 
they were being tracked by SAM radar, when^ missile was 
launched^and when a missile was close, respectively. Pilots 
were instructed that they were likely to be shot down if Chey 
flew above about 110 m for more than about 10 sec. Each- 
pilot carried out this exercise six times. The SAM and AAA 
threats served to ensure that performance would be strongly 
influenced by visual factors, since low-level flight is 
heavily dependent on out-of-cockpit visual input. Moreover, 
since the target could not be seen from low altitude, visual 
judgments were important in judging when to pop up and in 
correctly placing the aircraft during the dive. Bomb aiming 
and release were entirely visual and under the pilot's con- 
trol. The manual bomb delivery mode was employed so that 
accuracy was entirely determined by the aircraft's trajectory 
and speed at the moment of release. We measured the percent 
time tracked by missile radar, number of tim^s shot down, 
number of crashes, number of bomb delivery hits and near 
misses (yithin 36 m radius), altitude and heading variations, 
variability in release height, and release "G." 

Restricted visibility landings were carried out as de- * 
scribed [in Kruk et al. • 1981]. In brief, pilots .were told 
that airspeed was 120 kts (62 m/s), glide slope 2*5°, and 
that the initial ILS (Instrument Landing System) flight path 
was randomly set to land 61 m to left or right of centre. 
Visibility was nominally 460 m so that the runway was in- 
visible early in the* approach. Pilots were instructed to 
land cm each approach, and each pilot completed six landings. 
We measured the number of crashes, and the distance from run- 
way threshold at which the first visual flight correction was 

made. ' 

Visual tests comprised superthreshol^ velocity discrimi- 
nation of a radially expanding floW pattern, manual tracking 
of both motion in depth and motion in the frontal plane, moi 
tion thresholds and contrast thresholds for a moving square, 
and a static sinewave grating. 

' Landing and formation flight performance correlated with 
both manual tracking and expanding flow pattern test results. 
Pilots who were better able to discriminate different rates 
of expansion of the test flow pattern athieved a greater 
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percentage of hits and near mi sses in the low-level flight 
and bombing task. Aircraft flying grades for student pilots 
correlated with expanding flow pattern test results and with 
manual tracking of motion in depth. 

Implications for Simulator Design god Pilot Selection Tests 

Of the 13 threshold and superthreshold te,st measures, 
tests of superthreshold visual sensitivity to motion corre- 
lated most strongly with simulator and aircraft flying per- 
formance. This indicates that intersubject differences in 
the physiological processing of motion were an important de- 
terminant of intersubject differences in flying performance 
on our particular tests. Our findings suggest that motion 
tests (e.g., the changing-size tracking test and superthresh- 
old velocity discrimination) might, perhaps, be usefully 
added to the present, mainly static, battery of visual tests 
used in selecting flying personnel. Our findings also sug- 
gest that the transfer of training from simulator to aircraft 
flying tasks, including low- level flight , Restricted visibil- 
ity landing and formation flight, might be enhanced by placing 
more emphasis on improving the accurate, artifact-free repre- 
sentation of motion in simulator visual displays as compared 
with the historical emphasis on improving static spatial fac- 
tors such as resolution and number of edges. 

Ongoing Research 

In cooperation with Yuma Range and Williams AFB we are 
currently working on extending the simulator studies to in- 
clude performance measures in aircraft, with the aim of find- 
ing whether the visual factors we have identified as important 
in certain simulator flight tasks are also important in the 
air. 



Some Suggested Experiments 

1. Define the rqle and important parameters of expanding flow pat- 
terns in judging the direction of self-motion (e.g., spatial frequency 
and dynamic requirements) . 

2. Find whether visual sensitivity to stereo motion-in-depth is 
Involved 'in low-level flight in fixed-wing aircraft and helicopters. 

3. Define the role of visual sensitivity to changing size in 
low-level flight. 

4. Define the role of visual sensitivity to texture shear in 
recognizing objects and judging distance in flying environments, in- 
cluding nap-of-the-earth operations. 
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APPENDIX D • 

LOCOMOTION THROUGH VISUAL SPACE: OPTIC FLOW AND 
AND PERIPHERAL VISION / ' ' 

Ralph Norman Haber 



thfii design of flight sitBulation d for low-level flight must 

necessarily compromise a realistic presentation of the full visual 
scene unfolding before and beneath the pilot. It is simply not possi- 
ble to provide even reasonable verisimilitude of detail oyer the entire 
visual field of the pilot at rates approaching the real time demands of 
flight. The faster the velocity and the lower the altitude, the more 
severe the compromise. And the compromise is extreme for low-veloeity 
flight as well, though not because of real-time limitations^ Of cburse, 
low-level flight simulation has been practical for a number of years, 
with the exercise of many compromises in how visual information is pre- 
sented to the pilot. Most of these compromises .have resulted from hard- 
ware limitations, and not necessarily including art analysis qf how the 
human visual system processes information about locomotion • through space 
In order to examine the impact of these compromises, especially in 
a context of the information human beings use to perceive visual space, 
the following is a brief visual task analysis that relates sources of 
visual information available in different flight tasks, especially, 
hose tlyit can or do benefit from training in flight simulators. 

Visual Flight Task Analysis 

Sources of visual information available to a perceiver when view- 
ing a natural scene while moving through it can be grouped into three 
general and independent classes: those contained in the pattern of 
/disparities that exist between the two retinal images; those contained 
/in the patterning of the continuous relative displacements of projected 
light on each retina tfcat ofccur as the plane moves; and those contained 
in the perspective transformation of the light reflected from surface 
features as those features are projected on each retina. (I am exclud-\ 
ing and ignoring several potential local sources of inf option, prob- 
ably of little importance when looking at natural scenes, but perhaps 
used in the laboratory or other impoverished circumstances. Examples 
are accommodative and convergence corrections of, the eyes as voluntary 
fixations shift from near to far or vice versa. Y Each of these three 
classes provides information about the layout of space information 
that can fulfy specify the arrangements of space, such as surface 



• • ; 73 



* ori6=nt«a!cicpi and scale, observer position and change in position rela- 
* , tive to feh6 'surface, and location of all objects attached to the ground 
surface. m 

But the specification of the layput of spaee is quite different 
for each class of sources, with substantial differences in how the in- 
formation is processed. For example, information produced by observer 
motion is available orily when the observer in fact „ moves fast enough 
relative to his 'distance to the terrain surface so" that the differences 
in displacement of edges on the retina are above threshold. There is 
.little motion perspective information availabi^ to a hovering helicop- 
ter pilot, regardless of the nonuniformity 'of the terrain, nor to a 
bomber pilot at 50,000 feet even at supersonic speeds when the terrain 
is relatively flat. Further, binocular disparity information is avail- 
able only when the observer's point of regard is close enough to the t 
s surfaces being looked at sp as to be above^stereoacuity thresholds. 
Since the two retinal images produced by a scene are virtually identi- 
cal when the scene is more than several hundred meters distant, insuf- 
ficient disparity is present to be detectable and processed. Hence, 
in theory, for most flying tasks, adding a second eye does not add much 
information beyond the increased field of vie>w. This conclusion does 
not hold in theory for most helicopter flying tasks or for air-to-air 
Refueling, for example, si^ce for those tasks visual distances are well 
within stereoacuity limits so that disparities are above threshold. 
In practice, some low-level flight tasks even up to several hundred 
meters may benefit from binocular disparity information, but this is 
yet to be (Jemonst rated. 

While motion-produced visual information requires a moving ob- 
server and stereopsis requires the surfaces of the scene to be nearby, 
Information from the perspective transf ormation^ of surface features is 
theoretically always available, even to a one-eye stationary observer. 
All that is needed are perceptible surface features such as textured 
surfaces and objects; contrasts in luminance or color between ddjacent 
^objects, between objects and the ground, or between , different facets 
of each object; and of course contrasts produced* by shadows or variar- 
tion In surface orientation relative to the sources of illumination* 
s The grain of these features differs, so at- near distance the texture 
of the ground substance itself may be above threshold, whereas the 
same scene viewed from a great distance has a texture composed of the 
spacing of ground obj-ects, such as trees or hills. 

The above discission of the kinds of visual information available 
from natural scenes suggests important restric^i^is when considering 
flight simulation for various flying tasks. For example, a stereo- 
scopic display is probably important for helicopter pilot training, 
but is not likely to provide any useful visual Information for any 
fixed-wing aircraft operation tasks (except perhaps air-to-air refuel- 
ing) because all such tasks place the pilot at too grreat a distance 
from other visible surfaces or objects. Some careful work is needed 
to justify this conclusion, since it may turn out that there is suf- 
ficient stereoacuity available for some Low-level flight tasks. The 
evidence presented by Regan /nhppendix C of stereomotion-processing 
suggests the (ieed for binocula/ displays to fully portray all of the 
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information produced by observer motion; but for this, too, it needs to 
be demonstrated that sterejomotion acuity is present even at the dis- 
tance typically employed in low-level flight. 

While a stereoscopic display in a helicopter simulator may be 
useful or even important, accurate display of surfrfce features of 
helicopter flying is critical. When so much of the 'flying is at very , 
low speeds, and ,of tgnat no speed at all, movement produced changes 
are unavailable or unl^f ormative . In the absence of a stereo display, 
the only £nformati#* wiTl>*©iae from the simulation of< perspective • 
trans formtfU^ps of light reflected surface features; arid if the fidel- 
, ity of those surface features is poor, if only^a few features are 
- present, or J,f their resolution is inadequate Simulator performance 
' will necessarily be unrepresentative of actjiai flying performance. 

This analysis suggests- quite different 7 visual specifications for 
simulation at low-level flight. The most useful source of visual in- 
formation arises from observer-produced motion. Surface derails are 
* still important in the simulation, but since the pilot is capable of 
"moving and maneuvering so rapidly, it Is the motitfft information that 
Is so predominant. Thus', improving fidelity of surface texture, es- . 
p^tially 6f fine-grain texture, may not improve low-leval flight simu- 
lation at all, whereas providing visual 'information that is easily 
processed for relative displacements over time across the retina would 
seem to j>e essential.^ Westra, Slmon^ Collyer, and Chambers (1981) 
reached the same conclusion base3""up^n^^ 

of a number of surface add motion variablei^foO^ar^ Might 
performance, also' a low-level flight task. 

Given th^. focus of this report /on low^le^el flight, /the purpose 
of this brief £aper is to examinVfrotne o f the information sources that 
are produced, by the pilot's movetnenttfi^ough^pace^^ I' -especially want 
to single out one subclass of. these sources , >that based on optic flow 
•^patterns. This' focus has potential interest because attending to such 
information sources may actually reduce rather than> incr^se the # de- 
mands on competing capacity, and because they ar<r likfcltf 'to be especi- 
ally informative of the layout of viaual space. Some flugge stive ex- 
periments to be done on a wide-angle flight simul^tor^^ sketched at 
the end. 

* " ■ 

A Division of Labor— Central and Peripfi&Vift Vision 

We tend to look directly at the important sources of information 
in visual scenes because doing so images that information on the cen- 
tral high-resolving pax;t of the retina. But the fovea can cover only 
about one-ten-thousandth of the area of a visual scene at any one 
moment, the rest falling \>n the peripheral retina of lower resolving 
power. While we have learned a great deal about information process- 
ing in central vision, much less is known about how peripheral stimu- 
lation's impaired when only central vision is permitted^ Particu- 
larly, it seems likely that tasks involving orientation in visual 
space and especially locomotion through visual space depend upon 
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information from the full scene information available only to the 
peripheral retina. 1 

While the above comments are couched in a somewhat intuitive form, 
several recent theoretical developments have helped formalize the in- 
tuitions. The most important of these is the , notion that visual func- 
tioning maybe served by two somewha£ independent visual processing 
systems, one concerned with object Precognition and identification— 
the "what" of perceptual experience; and the other concerned with 
spatial localization and orientation — ; the "where 11 question (e.g., 
Schneider* 1969; Held, 1970). Such a division of functioning coin- 
cides closely with the differential sensitivities of the central versus 
peripheral retina, and with the differential degree of optical resolu- 
tion over these areas. It also relates to differential, loss of func- 
tion th&t occurs with disturbance to different brain structures (e.g., 
Poppel et al., 1973), suggesting some independence to processing of 
what and where information . Finally, it has been shown (e.g., Leibo- 
witz et al., 1979) that while questions of identification are criti- 
cally dependent upon .both stimulus energy and the optical quality of 
the retinal image, spatial orientation processing is relatively inde- 
pendent of energy and image of quality as long as the scene is visible 
at all. * 

A substantial part of the task facing a pilot during low-level 
flight concerns having precise visual information about the location 
of the ground in relation to the plane as it cha nges from moment to 
moment. From the distinction presented above, it would seem likely 
that such information is imaged over the entire retinal surface and 
not concentrated only over the fovea, and is not dependent upon either 
high optical resolution or intense energy. Both of these conclusions, 
if true, have obvious implications for the design of low-level flight 
simulation displays. 

Spatial Orientation Information Available * 
to a Moving Perceives 

Perspective geometry provides a description of changes in the pro- 
jection of light to an observer reflected from objects and surfaces 
as the observer moves (see Haber, 1979, for a brief review). The sim- 
plest circumstance, called motion parallax, is when there are only two 
points in the scene reflecting light: in such" cases, the light re- 
flected from the two points shifty position on the retina in propor- 
tion to their relative distance from the observe;:. Thus, the amount 
of relative change over time on the retina can provide information 
about relative distance from the observer — inforioation that is not 
available to a stationary observer. While motion parallax is a simple 



Empirical investigations have rtot shown much effeqt of display field 
of view on tasks such as landing. In contrast, with some tasks, such 
as <aerobatic maneuvers, field of view has been found to affect per- 
formance (see Hennessy et al., 1980). 
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circumstance, and is considered one of the basic monocular cues to 
depth, much more interest has been focused on its generalized exten- 
sion, in which all of the projected points of light move relative to 
one another as an observer moves through a visual scene. Gibson 
(1950) referred to this as motion perspective and emphasized that the 
information covers the entire retinal projection surface and is noj; 
restricted to merely a few points of light in relative motion. Thus, 
when an observer moves, a continuous gradient of perspective is pro- 
jected onto the retina, with the rate of change at each point provid- 
ing a mapping of the contours of the terrain being traversed, Gordon 
(1965) and morfe recently Koenderink and van Doorn (1975, 1976) have 
provided a detailed breakdown of the geometrical information available 
at 3 projection surface to a moving observer. Nakayama and Loomis 
<197A) have carried this work further by suggesting the kinds of 
neural mechanisms that could encode this gradient — neural mechanisms 
v of the same order as those t£at account for binocular disparity en- 
codings. Ullman (1979) has incorporated these features with a mor*e % 
general model of motion perception, and Lee and Lishman (1977) have 
reported some psychophysical experiments on the visual control of 
locomotion using motion perspective information. , 

One aspect of motion perspective discussed in detail by Gibson 
and others since refers, to opt^c flow information. When looking 
ahead, light ^reflected from the point in th^scen^>t?oward which ^he 
observer, is moving remains centered on t;h^^t±rra, only expanding in 
size as it gets closer.' All other reflected points flow across the 
retina toward the periphery, with a relative rate of flow proportional 
to the distance of the reflecting surface to the observer. In this 
way, the flow pattern represents the layout of the scene, with the 
aiming point of the moving observer being indicated by a zero flow 
vector. Since observers usually look at tftat. toward which. they move, 
this means that information about \ the direction of observer motion is 
available to central vision. But the 1 flowing part of the flow pattern 
covers the entire retina, and not only can specif direction and ve- 
locity of the observer, but more* important , the location and irregu- 
larities of the terrain. None of. this specification need be projected 
to central vision. It is likely that such optic flow patterns are 
the most important source of information about general terrain char- 
acteristics and the observer's, orientation and location over that 
terrain. Because optic flow does not depend upon cen tral vision, it 
may be handled entirely by the "where" system, ancJrtfP so, flow can be 
represented by <a 'simulator display with low-rasolution, low-intensity 
elements. 

I have three further notes before suggesting some experiments. 
When -an observer moves relative to the terrain, a system with high 
temporal resolution is capable of representing each projected point 
of light as a unique point. The human visual system, however, has a 
relatively low temporal sensitivity, so that a moving point of light - 
is perceived as a streak or blur line (Smith, 196a). Much of the op- 
tic flow has to^be streaks, especially in the periphery of the retina. 
Harrington et al. (1980) have shown that perceivers are able to judge 
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orientation quite accurately from optic flow that was entiife^y. blurred, 
even out of 75 degrees from the fovea. ' 

My second note concerns the effects of psychological or physio- 
logical stress on the processing of "what" and "where" information, 
and especially on the relative weights placed on central and periphery 
information. Mackworth (1965, 1976) reported studies that suggested 
that under stress or high information loads, identification process- 
ing becomes tunneled, so that information picked up peripherally is 
less adequately processed. His studies did not investigate location 
or orientation information processing, so it is not clear whether 
stress produces tunnel vision for "where" as well as "what" informa- 
tion; though it is possible tha,t the periphery does not suffer 1 any 
functional loss for processing of optic flow information. It is im- 
portant to know how the: details contained in the flow are processed: 
(a) which provide information about the objects on the terrain, as com- 
pared to how the flow itself is processed, and (b) which specify the 
orientation of the terrain. 

My third point is a more general one. We as yet do not know all 
o£ the sources of visual information that are available, and 
those, we do not know which ones are used in what contexts. T^us;v% 
one of the goals of research programs on space perception, even one 
focused on simulation training, has to be to examine the psychophysi- 
cal relationships between sources of information and subsequent per- 
ceptual performance-. This need is powerfully illustrated by the recent 
work of Owen (e.g., Owen et al., 1981a,b), who has shown that it mat- 
ters greatly whether the visual variables are defined in terms of 
visual information specifying environmental surfaces or' in terms of 
specifying the relations of the perceiver to those surfaces. In 
Owen's experiments, it turns out that the latter specifications are 
the ones which predict performance, suggesting that the psychophysical 
relationships include the perceiver in the equation. The important 
point is that there are many ways to define and specify each of many 
different optical variables, not all of which are equivalent. Only . 
by psychophysical research can we determine which variables have what 
properties. It cannot be done simply by a geometric analysis in kn 
arm chair. v \ 



Sdme Suggested Experiments 

With, these questions and assumptions in mind, I have- sketched 
out some research problems that should be addressed and, in some 
cas6s, have provided more details as to procedure. While my focus 
is primarily on the use of visual" information that falls on the 
peripheral retina during low-level flight and therefore in how to 
present such information in simulation of low-level flight, expanding 
our knowledge in this area will correct deficits in current theories 
of visual perception and could be of great practical value in design- 
ing the arrangements of visual displays such as instrument panels, 
monitors, and warning devices ; and in designing general visual simu- 
lation equipment , and the programs for displaying simulated scenes, 
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not only for low-level flight but for tasks like driver training and 
for improved methods of training target detection. 

What is needed are studies directed at demonstrating the aspects 
of the visual display actually used in various locomotion tasks. 
Since we need to know this within the context of low-level flight 
simulation, such studies require the use of a wide f ield-of-view 
simulator programmed for low-level flight. For some of the experi- 
ments sketched below, an additional clqsed loop control is needed, 
so that the monitored direction of gaze of the subject's eyes is used 
to vary the content of the display. Hence, when looking straight 
ahead, the details of the optic flow pattern seen through the side 
windows would be of low resolution, but if the pilot turned his head 
to look to the side, the resolution would sharpen to correspond^ to 
the* improved acuity of central vision. Sych. on-line monitoring of 
gaze direction and closecl loop control of display resolution is not 
necessary for all of the experiments, but it is critical for some 
that should be done . 

The general procedure involves pilot-subjects (perhaps including 
Subjects with little flight experience) flying in a wide-view simu- 
lator on various low-level flight tasks, such as terrain avoidance 
while maintaining a limited altitude, reconnaisance and target acqui- 
sition, or more specific tasks such as flying a prescribed course or 
maintaining a fixed altitude above the ground. Given these types of 
tasks; various stimulus changes could be evaluated as to their effect 
on flight performance. 

One set of studies involving stimulus manipulation might provide 
a mislocation of the zero point of the pptic flow vector, so that the 
plane's flight path was not always aimed at the zfcro point, even though 
the rest of the flow pattern was correct. If pilots use this informa- 
tion, substantial disruption in flight performance (and perhaps dis- 
orientation and discomfort) should occur. A related manipulation 
'might occlude the zero point in the flow pattern so the pilot could 
see it, though he could see all else in the scene. Further, along 
the same lines, we could manipulate how far' ahead of the plane optic 
flow information is useful in relation to velocity. When flying^at 
high speed, pilots gain little useful information directly in front 
of the plane. In effect, the pilot has to focus- well ahead. This 
tunneling of vision produces more reliance on optic. flow because less 
detailed pattern information can be picked up. 

A different set of studies should look at the content of the> 
flow pattern ijtself. 

& Conceivably the flow is processed entirely in the peripheral 
retina, and conceivably it is handled -by neural circuits concerned 
only with motion and v not with pattern or edges. In that case, it 
would be less important how flow is represented in the, display, and 
considerably lower resolution could be used without loss of perfor- 
mance. Again, the same sorter of tasks can be used to evaluate per- 
formance as a function of the content and the resolution of the flow. 

A third get of studies could look at an interaction of flow pat- 
terning with detection of specific targets as the flow passes. If 
flow is merely relative movement, how do we also detect the presence 



&0 



79 



of a sought-after target object? When looking straight ahead at the 
zero vector point, what are the minimal size, contrast, color, and 
context characteristics of objects displayed in ,the optic flow neces- 
sary for detection as they stream past the observer? Presumably, a 
sought-after or surprising object, once detected, causes an eye move- 
ment toward it away, from the zeto point, but what does it take to 
produce detection and reorientation? 

These studies constitute only a small subset of possible ones 
that could be done on flow and peripheral information pickup. But 
they are a start, and they can be easily expanded once the prograins 
"and procedures are set up. None # of these studies are ground-breaking 
in a theoretical sense, though they will allow us to test some of our 
hallowed theories. But they provide needed data for design of flight 
simulation displays, as well as general visual training procedures. 
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APPENDIX E 

DISPLAY DESIGN VARIABLES PERTINENT TO LOW7LEVEL ' 
FLIGHT SIMULATION 

Harry L. Snyder 



Introduction 

The main body and other appendixes of this teport contain analy- 
ses of known difficulties and problems in the visual simulation image 
content of low-level flight. Issues of pertinent visual scene param- 
eters are addressed, the importance . of many- visual parameters to pilot 
training and performance is noted, and suggested experimental activi- 
ties to define better the information content of the visual simulation 
are described.. This paper, off the, other hand, takes a different 'ap- 
proach, one complementary to the others. It does not address. the re- 
quirements for information content of the Visual scene, but rather 
addresses the requirements for the display generation equipment, in 
visually relevant terms. 

This appendix therefore serves two important purposes. First, it 
notes the pertinent hardware /software design requirements for visually 
compatible information presentation i_f the information is adequately 
defined and described in usable parametric terms. Second, it addresses 
some of the technologically limiting characteristics of visual dis- 
plays, thereby noting the limits of the visual information which can 
be presented. 

In the final analysis, if visual information presentation require- 
ments are found, through analysis or empirical experimentation, to ex- 
ceed current display, generation technology, then research and develop- 
ment requirements for simulation technology can be defined In a useful 
manner* On the other hand, if information requirements are far less 
than those of which* the simulation equipment is capable, then potential 
cos»t savings- and complexity reduction can be achieved. In either event 
it Is important to understand the relationships between visually perti- 
nent variables and measurable characteristics of the simulation 
equipment. 

Finally, this' appendix addresses briefly the issue of measurement 
and verification of the visual display in photometric and geometric 
ways directly pertinent to the visual characteristics of importance. 
Display technology has advanced rapidly, at a more rapid rate than the 
metrology needed to define and refine its image-producing specificat- 
ions. While some work has been done recently to recommend those 
techniques and criteria for adequate and accurate display quantificatio 
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universal application of these techniques is far from prevalent. Thus, 
the relationships between some of these measurement techniques and 
vision-related variables are summarized briefly. 

Display Variables Critical to Visual Simulation 

This section describes those display variables considered to be 
critical to the presentation of visual information in low-level flight. 
The format, density, and content of the information to be displayed is, 
described elsewhere in this report. Th^s section assumes the need for 
the visually compatible presentation of various types of information, 
and suggests minimal relationships which must hold for adequate pre- 
sentation of that information. Wherever possible, known data and ex- 
perimental results will be referenced. 



Display Field of View 

The fUeld of view (FOV) requirements for visual simulation have 
been amply shown to be task dependent. Take-off and landing tasks ap- 
pear to require only a limited FOV, on the order of 50 to 60 degrees 
or less. On the other hand, air-to-air combat , and by analogy low- 
level flight, requires visual scanning by the pilot and foveal acqui- 
sition of high spatial frequency information from various parts of 
the visual field typically available through the cockpit canopy. 
Studies have included parametric investigation of FOVs as large as 
180 degrees. Although statistical results have been limited, it ap- 
pears that the larger (e.g., greater than 150 degrees) FOV is needed- 
for reconnaissance, low-level navigation, and target acquisition ac- 
tivities, and perhaps for terrain avoidance. 

Translation of this requirement into realistic hardware presents- 
several problems. First, a single image, from a single projection de- 
vice, to cover at least 150 degrees is limited to either low-resolution 
electronic imaging or higher-resolution film-projection devices. For 
example, the JTF-2 visual simulator had a field of view of 160 by 60 
degrees, used a 70 mm film format, and had a limiting (high-contrast) 
resolution on the order of 3 arc minutes in the center of the FOV. 
Single CRT projection devices will have a maximum resolution on the 
order of 1,200 picture elements (pixels) per display width, and there- 
fore no better than 8 arc minutes limiting resolution. For this 
reason, the typical approach has been toward butted images from sev- 
eral CRTs, with the attendant problem of edge discontinuities or Image 
gaps due to registration errors between adjacent images. (The ad- 
jacency or segmentation problem is addressed below.) High-resolution 
elements (e.g., targets) can also be inserted from a second projector, 
and high-resolution "windows" in the total field can be slaved to. the 
pilot's head position to improve resolution in a part of the display. 

At the present timet the designer has a complex trade-off to make 
between FOV and resolution, increases in one generally causing de-' 
creases in the other. Since no quantitative data appear to exist on' 
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the effects of FOV on low-level flight control or navigation, it would 
appear that parametric research dedicated toward this detsenjiination is 4 
extremely important. Preceding .such experimentation, one should con- ; 
duct thorough analyses of the task and information requirements of 
the pilot in the mission phases of interest. As Semple et al. (1980: 
79-80) -pointed out, 

the value of a wide FOV may not have been apparent in the 
studies reviewed because the tasks did not specifically 
demand use of more than a central FOV. For example, air 
to air combat obviously requires a wide FOV. Therefore 
the decision on whether a wide or narrow FOV is necessary 
for training purposes must take into account specific task 
requirement^ and not be based solely on^ possible unwar- 
ranted generalizations from research results . . . from 
a limited number of flying tasks. 

The low-level navigation and target acquisition task is not among those 
which has been parametrically investigated, and therefore requires 
careful research on FOV trader-offs. 



Dynamic (Luminance) Range 

V 

Projection systems have a finite amount of luminous energy that 
can be distributed across ' the FOV. Increases in the FOV, obtained ' 
optically, will result in proportionately lower luminance levels, as- 
suming screen gains and optical efficiency to be approximately con 1 
stant. The only exception to this generalization is .in the use of 
multiple CRTs or other segmented means &f controlling the image lumi- 
nance per screen segment by increasing the number of sources rather 
than spreading the energy from a single source over a wider FOV. 

While the observer will visually adapt ^£o the mean luminance % 
level of the entire display, assuming scannirfg behavior such as to 
fixate foveally the overall field, reductions in luminance range will 
necessarily reduce the number of discriminable levels of luminance. 
More importantly, the contrast sensitivity of the observer will de- 
crease with decreases in luminance, thereby further reducing" the ob- 
server's discrimination of luminance differences and decreasing his 
cutoff spatial frequency, resulting in a loss of effective acuity. 

It would appear that the attenuation of luminance range is im- 
portant in that the contrast sensitivity function (CSF) shift is 
critical to # the perception of small details,, and perhaps to the per- 
ception of depth relationships. Thus, luminance requiremfmts can be 
stated only once we kn6w the requirements for spatial information 
presentation (resolution) of the image. Experiments to determine 
the resolution requirements are considered important for this as 
well as other reasons. 

Existing research* results permit ,us to recommend, a limiting 
resolution of no more than 3 arc mljuites for such activities as 
target acquisition in an air-to-ground search (Humes and 
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Bauerschmidt, 1968) . The fifuthor knows of no comparable data for per- 
formance effects attributable to resolution In tasks such as low-level 
navigation and terrain following. Again, a task analysis leading to 
Information reqOlrements might help. Ofiher experimental techniques * 
based upon scene spatial filtering and observer static target and pb- 
ject recognition might also be helpful. 'Simplified fclosed-loop flight 
control simulations with varying image resolution may also sferve to 
provide useful infQrmation on the requirements for display resolution 
and therefore dynamic range. ( • 

\ v ■ * 

MTF ' • 

tfhe preceding discussion related resolution 'requirements to dy- 
namic range. A more useful measure, which combines both luminance 
range and resolution is the modulation transfer function, or MTF*.. 
In this approach, the output modulation (or contrast) of the display 
is determined for all spatial "frequencies , assuming a constant input 
modulation (M) . Thus, the MTF specifies the modulation the display 
systejn is capable of delivering at any spatial frequency , assuming a 
unity modulation input. The engineering convention of describing 
"shades of gray" as 20.5 increments in luminance can be converted to 
*. equivalent modulation by the following formula: 

/ ■ « c v / , log (-(1+M)/(1-M)) m 
N (shades of gray) « 1 + — & — * tt-~ UJ 

' ' " log 

i In recent years it has become recognized tha* the Specification 
\' of an MTF is more meaningful and pertinent than t He specification of 

merely a limiting resolution. The "limiting resolution" of an imag- 
ing system- is, in actuality, the spatial frequency at which the MTF 
of the display system crosses the CSF of the observer (Snyder, 1973). 
\ More importantly, the MTF also desc^Bes the ability of the display 

system to rende.r high contrast inputs faithfully at spatial frequen- 
cies below the limiting, resolution . As such, the MTF describes the 
ability of the imaging system to faithfully reproduce input contrasts 
at all usable spatial frequencies of the system, whereas the limiting 
* resolution specifies only the spatial frequency at which the system 

ceases to provide information to the observer, a measure of little 
Utility. 

Little research has been conducted to relate observer, performance 
to MTF variations for mai^y tasks, although data do exist for tasks . 
such as face repognitiQn and air-to-ground target acquisition (Gutmann 
et al., 1979; Snyder, 1974; Snyder et al. , 1974) . Assuming the low- 
level navigation and flight control tasks to be reasonably .related to 
air-to-ground target acquisition, then it would appear that a display 
systepi with an MJF at least equal to that shown in figure E-l would 
be desirable. Unfortunately, as figure E-l also illustrates, current 
technology is far from this capability. Research of a more directly 
applicable natur6, using flight control rather than target acquisi- 
tion as the dependent measure, is clearly indicated. 
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FIGURE E-l Recommended minimal dislRr system MTF for low-altitude simulation. 
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Refresh Rate 



Computer-generated displays are of finite bandwidth, and this 
bandwidth must be allocated between the number, of displayed picture 
elements (pixels) and the refresh rate. It is often feasible to 
reduce the refresh rate by using longer persistence display surfaces, 
■thereby achieving greater spatial resolution. However, with dynamic 
"motion typical of low-altitude flight, longer persistence displays 
would be subject to blur or streaming. Thus, a careful trade-off is 
needed between spatial resolution and temporal resolution (refresh 
rate). This trade-off also exists in film projection systems, where 
the shutter angle of the cine projector, in conjunction with its 
frame rate, must be selected to avoid image jumfk or 'strobing. 

The refresh rate is set by both flicker fusion thresholds and 
the image motion rate. As image motion- increases , typically at the 
bottom center portion of the display as a limiting condition, inade- 
quate refresh rates yield image breakup or a perception of image 
strobing (Snyder et al., 1966). Solutions to. the strobing problem 
are to increase the refresh rate; permit blurring, or decrease the 
angular rate of the image by decreasing the look-down angle. These 
trade-offs are quantitative and established in terms of the strobing 
phenomenon (Snyder et al . , 1966); further, the minimum refresh rate 
to avoid flicker can be specified by a Fourier analysis of the tem- 
poral luminance distribution, which includes the refresh rate con-, 
volved with the persistence of the display system (Snyder, 1980).. 
Reductions in refresh rate 'are helpful in bandwidth allocation. A 
60-Hz rate, while standard, may not be needed. 

In general, then, the needed specification for refresh rate de- 
pends largely upon knowledge of the field of view, the maximum simu- 
lated aircraft velocity and altitude (to obtain V/H) , and the number 
of pixels'in the display requiring refresh every .f tame rate . No re- 
search seems warranted dn'this parameter except for verification 
studies once the FOV and resolution/MTF requirements are sat. 

Information Update Rate 

The display refresh rate is' established to avoid flicker, and must 
also be compatible with the, need to eliminate strobing, as described 
above. At the same time, there is often a .need to refresh, the display 
to change the state of information depicted on the display. Some in- 
formation rate updates are required by the strobing problem, but 
others are needed to achieve sufficiently accurate Information place- 
ment in the field, even though strobing would not occur with a lower 
update rate. Careful 'analysis of the rate of movement of displayed 
elements, along with the placement accuracy required, both spatially. ■• 
'and temporally, is heeded for the low-altitude flight regime. This 
analysis, in conjunction with the information requirements work de- 
scribed elsewhere, should produce information update rates compatible 
with the pilot's visual needs. 
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Virtual Versus Real Image * 

Two approaches are popular in visual flight simulation. In one, 
the image is created on a display surface such as a CRT and viewed 
through a collimating lens to produce an Image at optical infinity. 
With this approach, the pilot can move his head in translation with- 
out relative image 'motion ; ift addition, the accommodation of the eye 
is comparable to that required in actual flight to view the real 
world. In this mode/the pilot must shift accommodation realistic- 
ally between the visual scene of the projected display and the in- 
cockpit information, creating thereby appropriate visual loading and 
accommodation time lags. 

The second approach largely igno£S&-*this problem of accommoda- 
tion shift between in-cockp^t tasks and; optical infinity, and pro- 
jects a real image on. a (typically) spherical screen in front of the 
cockpit. The radius of ourvature of such screens is usually on the 
order of 6 to 15 feet, thus requiring some accommodation shift between 
.the cockpit displays and the -pro jected real world; however, the ad- 
vantages of this type qf simulation are freedom of distortion and 
separation introduced by the* segmented collimating optics or separate 
display surface. 

It must be realized that noncollimated display of scene informa- 
tion that would normally exist at optical infinity may lead to mis- - 
perception of the size of objects (Leibowitz et al., 1972; Roscoe, 
1951). In addition, movement of the pilQt's head with respect to 
noncoll limited" images mpy result in- misjudged distance perception 
(Gogel and Teitz, 1973), although it is likely that an* image distance 
In excess of 10 feet will minimize this effect. 

Whether the colllmated display has any real advantage, particu- 
larly for training, over the noncollimated, real-image display is an 
empirical question. Certainly, if the noncollimated display can avoid 
the image segmentation typical of many muitiple-itaage , pancake-window 
optics used In colllmated, wide-FOV displays, then the advantage may 
well rest with the noncollimated display. It would appear, as sug- 
gested by Semple et al. (1980)., that the decision may well be a mat- 
ter of econpmlcs rather than performance, although this writer recom- 
mends a direct experimental comparison between the two, using the same 
tasks and imagery, to determine if any significant difference exists. 

Image Geometric Linearity 

Perfectly colllmated Images will produce geometrically linear 
'displays if the display generator' is geometrically perfect. All too 
typically, however, the edges of the display suffer from slight, non- 
collimation, exhibiting a small amount of "swim" in the corners. 
This can be distracting to the observer, especially if the. image 
corners are butted with other images. The same problem can exist in 
a noncollimated image if sufficient care is not taken with the ana- 
morphic optics (among others) to correct for screen curvature , center 



of curvature, etc. XThe subject's head and the projector cannot both 
occupy the center of curvature position!) 

- A critical question is the extent to which noncollimation or 
geometric nonlinearity may cause performance or training differences 
by pilots. There are no data on the effects of display nonlinearity 
on even simple tasks such, as object recognition or text reading let 
alone on pilot performance during low altitude flight (Snyder, 1980). 
Such data are needed, but are probably not as critically related to 
pilot performance or training efficiency as are other variables dis- 
cussed in this appendix. 

Image Segmentation 

* The problem of alignment and separation of multiple-image dis- 
plays was discussed in a literature review. and analysis report by 
Kraft et al. (1980), who recommended experimental investigation of 
the segmentation and alignment ^problem. Tin a subsequent investiga- 
tion, Kraft and Anderson (1980) determine^ the ability of U.S. Air 
Force pilots to detect joint-width differences, misalignments, and 
rotation^ among image portions. This study was conducted with static 
daylight scenes, and the subjects were.not required to perform any 
complex, closed-loop task by reference to the scenes. 

Kraft and Anderson (1980) found that joint widths of 15 arc 
minutes or more masked vertical displacement errors up, to 1.4 arc 
rainut&s. They also found that joint widths of less than 15 arc min- 
utes ted to more critical tolerances of rotational errors. 

Whether these acute judgments by their subjects would be upheld 
in more complex tasks, such as low-altitude navigation and terrain 
following, is an empirical question. It would seem doubtful that the 
heavily loaded pilot would be as. sensitive to such small errors .in 
the visual display under such conditions. Further, it may well be 
the case that detection of small image segmentation and alignment 
errors may have no impact on the training utility of the simulator. 
At the. present time, no useful dataj exist on this point. 

Jitter 

" Many CRT displays exhibit substantial vertical jitter, a field- 
to-field or frame-to-frame vertical movement in the image caused by 
instability of the vertical deflection circuitry. Unfortunately, 
such jitter is often greater than the spot size of the display, 
thereby causing a reduced vertical resolution or a high-frequency 
roll-off of the MTF in excess of what spot-size limitations would 

dictate. . , _ , , 

It is quite feasible that this jitter, while not detectable di- 
rectly causes many of the visual fatigue symptoms experienced by 
users of visual display units (usually CRTs) in office environments. 
If so it is also reasonable to think that jitter may cause visual 
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fatigue (tiredness, headaches, etc.) in the simulator pilot after sub- 
stantial time in the simulator. 

While jitter is measurable, it is often not carefully controlled 
in CRT deflection design. Thus, many solid state, flat panel dis- 
plays are judged to be better in image quality by^some observers due 
to their lack of jitter. . ' 

. No experimental data currently exist on the relationship between 
jitter amplitude and frequency and objective performance measures of 
observers. Further, no data exist on the effect of jitter variables 
on subjective reports of visual fatigue, although anecdotal stories 
have been recorded (Snyder, 1980). A parametric study of the effect, 
of jitter on visual performance is clearly warranted. In the absence 
of such experimentation, careful measurement and control of jitter 
should be of prime concern in the design of any visual simulation 
system. f\ 



Color 

The use of color in visual simulation for training has, been in- 
vestigated in at least two studies. Chase (1970) investigated the 
effect of color on training for approaches^ and landings, and obtained 
small but positive differences in favor of color. On the other hand, 
Woodruff (1979) found no differences in either learning rates or 
final performance using color versus black-and-white TV displays. 
Similar results have been found in air-to-ground target acquisition 
tasks. Snyder, Greening, and Calhoun (1964) found no differences in 
target acquisition for color versus black-and-white in a film projec- 
tion simulation, while Fowler and Jdties (1972) found no differences 
in an air-to-ground TV simulation. 

It should be noted that the atmospheric attenuation of color is 
quite rapid, with very little color remaining beyond a few miles of 
clear atmosphere (Middleton, 1952). With the exception of simulation 
of cultural objects, such as landing and runway/ taxiway lights which 
are intrinsically color coded, there appears to be little need for 
.color simulation, although pilots report that a simulator display 
with color is very pleasing (Semple et al., 19803"? 

It appeats that parametric experimentations on the requirements 
for color in low-altitude flight is urgently nee9^, not because of 
the aesthetics involved, but rather because of the economics. Image 
generation in color is usually more complex and expensive, as is the 
design of optics that avoid chromatic distortion. Further, the pos- 
sibility of Yalse depth perception, due to chromostereopsis, must be 
avoided. If color does not aid in either performance or transfer of 
training, then there appears to be no noncosmetic requirement for 
such. On the other hand, color enhancement might provide improved 
training efficiency, but this determination is yet to be made. 
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Stereo Imaging 

The need for stereoscopic display for flight simulation has been 
considered by many, but never implemented or evaluated experimen- 
tally. On the negative side, one can argue that most ^of the visual 
information pertinent to the flight environment occurs at distances 
on the order of several hundred feet or more, and that no binocular 
cues (e.g., convergence, retinal disparity) or accommodation cues/ 
exist at thai otical distance. Conversely, the existence of stereo 
channels (see appendix C) creates some concern for potential distor- 
tion of the visual *cene without presentation of stereoscopic infor- 
mation which might be above threshold for these channels. 

While this issue cannot be adequately resolved at this time due 
to lack of a complete understanding of the relationship between stereo 
channels and low-altitude flight performance, the issue certainly 
warrants further analysis and perhaps experimentation along the lines 
presented in appendix C. 

Measurement Issues in Display Variables 

In tluTpast, specifications /or visual simulation systems have 
been incomplete in that theyXhave not contained parameters and meas- 
urement procedures pertinent \o the observer's visual "< ul " m ^- 
For example, specification of video bandwidth, line rate and maximum 
.video voltage may define the limiting performance of the display, but 
they do not define the luminance range, the MTF^ the actual limiting 
resolution, the line pairing or interlace accuracy, or other vision- 
related characteristics. What is needed, of course, is a set of meas- 
urable displayed image variables, the required levels of- which are 
minimum for adequate visual performance of the pilot. In addition, 
there must be specified the operational procedures by which quality 
control of these characteristics' is maintained in both acceptaj-" 
testing and subsequent maintenance. To date, only a few milf* 
operational systems have been purchased and maintained under 
procedures, and no simulator systems have had to meet stringenl 
pe rtinent design requirements. ^ aonriii>[inf 

* fhe~following paragraphs describe but four of these measurement 

categories. While others exist, it is the proper province of the 
system designer to specify both the critical design parameters and 
the procedures by which those .parameters will be measured and main- 
tained. The following is therefore suggestive, and not definitive. 



Radiometric/Photometric Control 



Many CRT and flat panel displays are stated to provide certain 
levels of luminance (albeit often called "brightness") under maximum 
signal conditions. Some contain specified luminance levels for zero 
voUage input. Unfortunately, two problems exist in these «Pecifica- 
tions! First, the maximum luminance level is not achieved in a CRT 
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without some loss of resolution due to spot bloom; second, the "black" 
level is never at zero luminance due to scattering, flacepla^e reflect- 
ance, and other conditions. Thus, the obtained maxirtiimi modulation is 
always less than unity, and the true amount of achievable modulation 
is never stated. 

1 .While the problem is somewhat different for a solid state dis- 
play, the maximum luminance is often dependent on the number of 
pixels turned "on" and may'also vary with "half-select" voltage char- 
acteristics of the device, an issue rarely addressed in the perfor- 
mance specifications of the device. 

Recent research has resulted in recommendations for radiometric 
scanning of the display device under known input conditions. A 
scanning monochromator id used to obtain radiant energy per unit 
wavelength, which is then converted to photometric power and ulti- 
mately to luminance if that is of interest to the user. Accuracies 
are obtainable of better than 1 percent, and dominant wavelength can 
be measured to within 2 nanometers. CIE coordinates can likewise be 
calculated from the radiometric scans to within .002 in x and y (Far- 
ley and Gutmann, 1980). 

These measurement techniques are particularly valuable in color 
research and in chromatic control of the display when constant lumi- 
nance or constant brightness is needed. Further, computerized scanning 
and calibration is achievable on a scheduled basis, using self- 
calibrating radiometric equipment and programmed inputs to the display 
device. Outputs from periodic calibration can be used to define re- 
quirements for adjustment, maintenance, or replacement of display 
components. In particular, since the luminance and chromatic drift 
of many CRTs is as much as 10 percent in less than one hour, periodic 
measurement is needed during research activities and perhaps during 
simulation tests. 

i 

Measuring" MTF 

The static MTF of a display can most easily be measured with 
either a square-wave or a sine-wave grating inputted to the display, 
and the display surface scanned microphotometrically or microradio- 
metrically with a slit scanning microscope. The scan, taken over 
several cycles, is then Fourier analyzed to determine (1) the modu- 
lation at the fundamental frequency, (2) the spatial frequency of 
the fundament al7 to check for magnification errors, and (3) any har- 
* monic distortion as evidenced by inappropriate luminous power in \ 
higher harmonics. Since digitally addressed displays cannot reli- 
ably and accurately reproduce a sine wave at high spatial frequencies, 
the most convenient technique is to input a square wave, followed by 
correction by a factor of 4/pi to obtain the equivalent sine-wave 
modulation, assuming system linearity. 

Systematic inputs of varying spatial frequencies, followed by 
plotting of the modulation in the Fourier spatial fundamental, yields 
the MTF curve. This procedure^should be done in both the vertical 
*iod the horizontal display dimensions to measure both MTSs, as most 
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displays are anisotropic. In addition, when a raster scan is used, 
the luminous power in the raster can be determined in this fashion 
with a constant (gray) image input\signal . If any line pairing ex- 
ists in the raster, the Fourier line spectrum will show a reduced 
modulation at the raster fundamental frequency and more modulation 
at higher order harmonics. These and other measurement techniques 
are described in Snyder et al. (1974) and in Snyder and Maddox 
(1978) . Dynamic MTFs are more complex and are critical to this 
problem, but need not be discussed here. 

• 

Measuring Jitter 

The measurement of jitter requires careful measurement in the 
temporal rather than in the spatial domain. Jitter is most easily 
measured by placing the object plane aperture of the microphotometer 
at the edge of a raster or image element line, and then sampling the 
radiance or luminance output during successive refresh cycles. Vari- 
ation of this output, for a constant input, denotes jitter (or lumi- 
nous noise). Zero-lkg sampling can be most easily obtained by record- 
ing directly .from the dynode of a photomultiplier tube photometer and 
presenting successive* output levels on a storage oscilloscope. 
Measurements in rms luminance can be made directly from the oscillo- 
scopeimage if" the photomultiplier tube output has been calibrated. 
Details of this technique are described in Snyder et al. (1979). 

Dynamic Characteristics 
♦ 

CRT displays are subject to several forms of dynamic changes 
over time. Noted above were jitter, luminance Instability, and 
chromatic instability. These changes' are largely due to fluctuations 
in line voltage, heating of electronic elements within the display, 
and fluctuations in driving signals. Other image-related, dynamic 
characteristics of concern include chrominance changes due to mis- 
convergence of the color electron guns with the shadow mask, long- 
term drifts in luminance and therefore MTF, and long-term changes in 
spot size, which also influence the MTF and limiting resolution. 

Each of these dynamic changes can be measured with standardized 
spatial or temporal mlcrophotometric or microradiometric measurements, 
usually of the form described above. What is critical is that they 
are measured and are not simply assumed to be stable for the useful 
duration of the system. Since such degradations are gradual (often 
called "graceful"), they tend to go unnoticed, all the time poten- 
tially degrading performance of the user until total failure occurs. 

Measurement and drift are especially critical for 'those param- 
eters shown to be sensitively related to pilot performance. During 
research studies to determine display requirements, in particular, 
these measurements should be made systematically and periodically, 
recorded, and related to any trends in subject performance. In an 
operational training environment, such measurements would similarly 
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be made, recorded, and related to ^ortg-term trends in training effec- 
tiveness. Tq the extent that one mufet be careful in specifying opti- 
mum levels of display system parameters, one must also be careful to 
retain those levels during operational usage. 

r * 

Issues Warranting Research Consideration t 

Many variables relating to image content may require careful re- 
search to yield flight simulation displays adequate for efficient 
low-level flight training. However, specification of the image con- 
tent variables will be of little value if the displaced image is in- 
adequate to present the content with acceptable fidelity, consistency, 
and quality. For these reasons, the following variables may be 
critical in display specification/design and warrant experimental or 
analytical investigation as to their impact on performance and train- 
ing transfer for low-level flight: 

1. Field of View: The relationship between FOV (vertical and 
horizontal) and pilot performance for training effectiveness 
in low-altitude simulated flight must be determined, as this 
is critical to the design of a variety of display system 
characteristics . 

2. MTF: The effect of system MTF on pilot performance for 
training effectiveness must be similarly measured. In- 
creases in MTF are difficult and expensive , and pverspeci- 
fication is unfortunately possible, though greatly 
uneconomical. v 

3. Virtual versus Real Image: A direct comparison of collimated 
with noncollimated images should be made, using pilot train- 
ing performance as well as subjective ratings as criterion 
measures. Size, complexity, and cost considerations relate 
directly to this evaluation. 

4. Jitter: Some jitter (or other dynamic sources of image 
quality loss) will always remain in CRT-based systems. The 
key issue is the degree to which jitter must be controlled 
through expensive and complex design. No data currently 
exist, and empirical studies are needed to relate Jitter 
amplitude and frequency to pilot performance. 

These research recommendations are considered to be compatible 
with the image content recommendations of appendixes A through D. 
Determination of image content for pilot interpretation without ade- 
quate attention to display quality parameters will not result in any 
significant capability improvement. Similarly, improvements in dis- 
play quality, without regard for image content considerations will 
yield little gain. A compatible, integrated research effort that 
takes into account the perceptual content requirements of the trainee 
as well as the image quality variables is therefore seen as most 
promising. 
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APPENDIX F 



FLIGHT SIMULATOR WORKSHOP 

COMMITTEE ON VISION 
NAT I ONAL^'RE SEARCH COUNCIL 




Agenda 

8:30 - Welcome, 
9rbO - 11:00 



19 June 1980 

OT Orientation and Briefing 

OT Vision Research Overview: 

9:00-9:20 T-4G Collimation and Color 



9:20-9:40 
9: 40-10; 00 



Studies 

Visual Texture and Other 
Clues 



Vertical Object Cues 

10:00-10:20 Vlsual/Vestibular & Color 
Studies 

.10:20-11:00 The Use of Perceptual Data 
in Flight Simulation 
Design 



M. Rockway 



R. Woodruff 

^G. Buckland 
R. Engel 

R. Ktfllogg 
K. Boff 



11:00 - 


12:00 


ASPT Orientatioa^Tour 




12:00 - 


13:00 


Lunch, Officers 1 Club 




13:00 - 


13:30 


Hardware Developments 


B. Eriksoh 


13:30 * 


14:00 

> 


Future Direct/ons 


G. Buckland 


14:00 - 


17:00 


Committee on Vision: 





14 : 00-14 : 15 Introduction 

14:15-15:00 Visual Psychophysics : 
Size-Change and 
Disparity-Change Channels 

15:00-15:45 Recovering Surface Orien- 
tation: A Computational 
View 

16:00-17:00 General Discussion 



W, Richards 



D. M. Regan 



K, Stevens 



9:00 - 9:30 

9:30 - 11:00 

11:00 - 11:30 

11:^0 - 12:00 
12:00 - 13:00 
13:00 - 14:00 
14 iOO - 14:30 

14:30 - 15:00 
15:00 - 16:00 
16:00 - 
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Farmulation "bi-P-rpblem: Limitations 
to Low^Level Flight . 

Discussion of Important Visual Fac- 
tors for Sitoulating Low-L^vel Flight 

Listing, And Rating of Ef fectiveness 
of Visual Cues 

KYST Analysis 

Lunch 

Atterttional Factors 

Response Measure* <and Scan 
Strategies ~. 

Helicopter Flight Simulation 

• *» ...» 

General Discussion 
Adjournment 



W. Richards 



( 



D. Fender 

W. Schneider 

Y. Zeeyi 
R. Wri^t 



Participants 

K. Boff/ 

G. Buckland 

<%S. Collyer 

J. Christiansen 

K. Dismukes 

D. Fender 
F.» Gomer 
R. Haber 

P. lampietro 
T. Longridge 

E. Martin 



R. Kellogg 
D. Regan 
W. Richards 
W> Schneider 
H. Snyder 
K. Stevens 
H. Wilson 
R. Woodruff 
R. Wright 
L. Young % 
Y. Zeevi 



\ ■ ■ • 

• 



0 

1 o i 
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STEERING * GROUP ON FLIGHT S IMULATION 



Whitman Richards (Chair) » Departraen Jo# Psychology , Massachusetts In- 
stitute of Technology ^ " . / . A . gs 
Derek Fender, Jorgensen Laboratory of Information Science, California 

Institute of Technology . \ V 

Ralph Habet, Department of Psychology, University of\imnois 
Julian Hochberg, Department of Psychology, Columbia l^ivArsity 
Herschel Leibowitz, Department of Psychology, Pe/nsylvania State 



University ^ ^ . ... tl 4 . 

David Regan, Department of Physiology and Biophysiqs, Dalhousife Uni- 

^ versity, Halifax, Nova Scotia 

Harry Snyder, Department of Industrial Engineering and Operations Re- 
search, Virginia Polytechnic Institute and State University ^ 

■ I- . ■■' ' , ; ,> ' .- • 

Staff • ; 

Key Dismukes, Study Director 

Kent Stevens, Consultant 

Llyn Ellison, Administrative Secretary 




\ 
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COMMITTEE ON VISION 



Derek H. Fender (Chair), Jorgensen Laboratory of Information Science,, 

California Institute of Technology 
Anthony Adams, School of Optometry, University of California, Berkeley , 
Eliot L. Berson, Harvard Medical School ' „ . . 

John E. Dowling, Department of Biology, Harvard University 
Julian Hochberg, Department of Psychology, Columbia University 
Dorothea Jameson,- Department of Psychology, University of Pennsylvania 
'Ken Nakayama, Smith-Kettlewell Institute of Visual Sciences, Sdn 

Francisco * ' 

Luis M. Proenza, Department of Zoology, Uniyersity of Georgia 
Robert Sfekuler, Departments of Psychology, Ophthalmology, Neurobiology/ 

Physiology, Northwestern University 
H^rry Snyder, Department of Industrial Engineering and Operations 

Research, Virginia Polytechnic institute and State University 



Staff * 

Key Dismukes, Study Director 

Barbara Brown, Research Assistant 

Llyn Ellison,' Administrative Secretary \ 




